[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:24:39 dist-test-slave-6k22 krb5kdc[17854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:24:39 dist-test-slave-6k22 krb5kdc[17854](info): set up 2 sockets
Aug 06 20:24:39 dist-test-slave-6k22 krb5kdc[17854](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 20:24:40.283542 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:24:40 dist-test-slave-6k22 krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:24:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:24:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:24:40.494247 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34459
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:40.864957 17868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:40.865410 17868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:40.865585 17868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:40.872815 17868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:24:40.872959 17868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:40.873059 17868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:40.873157 17868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:24:40.873237 17868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:24:40.883457 17868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:40.885336 17868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:40.887117 17868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:40.907487 17874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:40.907613 17877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:40.907534 17875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:40.911486 17868 server_base.cc:1034] running on GCE node
I20260806 20:24:40.912608 17868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:40.914959 17868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:40.916319 17868 hybrid_clock.cc:648] HybridClock initialized: now 1786047880916232 us; error 79 us; skew 500 ppm
I20260806 20:24:40.916833 17868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:40 dist-test-slave-6k22 krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047880, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:40.920442 17868 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:24:40.922955 17868 webserver.cc:460] Webserver started at http://127.17.109.254:41991/ using document root <none> and password file <none>
I20260806 20:24:40.924059 17868 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:40.924216 17868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:40.924678 17868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:40.928767 17868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "658a329c10f04411a67d849dbb16b301"
format_stamp: "Formatted at 2026-08-06 20:24:40 on dist-test-slave-6k22"
server_key: "9c119e544bdc012b5779b54eaa9240df"
server_key_iv: "5004daaba6d47bd92215628bb441ea5b"
server_key_version: "encryptionkey@0"
I20260806 20:24:40.929653 17868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "658a329c10f04411a67d849dbb16b301"
format_stamp: "Formatted at 2026-08-06 20:24:40 on dist-test-slave-6k22"
server_key: "9c119e544bdc012b5779b54eaa9240df"
server_key_iv: "5004daaba6d47bd92215628bb441ea5b"
server_key_version: "encryptionkey@0"
I20260806 20:24:40.937462 17868 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260806 20:24:40.942778 17884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:40.944789 17868 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 20:24:40.945067 17868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "658a329c10f04411a67d849dbb16b301"
format_stamp: "Formatted at 2026-08-06 20:24:40 on dist-test-slave-6k22"
server_key: "9c119e544bdc012b5779b54eaa9240df"
server_key_iv: "5004daaba6d47bd92215628bb441ea5b"
server_key_version: "encryptionkey@0"
I20260806 20:24:40.945314 17868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:40.965466 17868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:40.971314 17868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:40.971830 17868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:40.997601 17868 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:34459
I20260806 20:24:40.997684 17936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:34459 every 8 connection(s)
I20260806 20:24:40.999904 17868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:24:41.006628 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 17868
I20260806 20:24:41.007190 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:24:41.007658 17937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:41.010998 17847 external_mini_cluster.cc:1428] Setting key b63bb47e61f62b017d539f6480b86af5
I20260806 20:24:41.026563 17937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301: Bootstrap starting.
I20260806 20:24:41.032349 17937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:41.034663 17937 log.cc:824] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:41.038800 17937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301: No bootstrap required, opened a new log
Aug 06 20:24:41 dist-test-slave-6k22 krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:41.048763 17937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658a329c10f04411a67d849dbb16b301" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34459 } }
I20260806 20:24:41.049444 17937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:41.049654 17937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658a329c10f04411a67d849dbb16b301, State: Initialized, Role: FOLLOWER
I20260806 20:24:41.051018 17937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658a329c10f04411a67d849dbb16b301" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34459 } }
I20260806 20:24:41.051398 17937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:24:41.051793 17937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:24:41.052035 17937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:41.054764 17937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658a329c10f04411a67d849dbb16b301" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34459 } }
I20260806 20:24:41.056241 17937 leader_election.cc:302] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 658a329c10f04411a67d849dbb16b301; no voters:
I20260806 20:24:41.057195 17937 leader_election.cc:288] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:24:41.057351 17942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:41.061074 17937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:24:41.064823 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:41.018580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46440 (local address 127.17.109.254:34459)
0806 20:24:41.020122 (+ 1542us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:41.020140 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:41.023005 (+ 2865us) server_negotiation.cc:407] Connection header received
0806 20:24:41.033572 (+ 10567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:41.033598 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:41.034349 (+ 751us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:41.034906 (+ 557us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:41.038419 (+ 3513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:41.040816 (+ 2397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:41.042841 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:41.044030 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:41.044266 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:41.051124 (+ 6858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:41.051184 (+ 60us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:41.051222 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:41.051292 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:41.054732 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:41.056437 (+ 1705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:41.056444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:41.056450 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:41.056740 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:41.057444 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:41.057449 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:41.057451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:41.058739 (+ 1288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:41.059198 (+ 459us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:41.061337 (+ 2139us) server_negotiation.cc:299] Negotiation successful
0806 20:24:41.061940 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1059,"thread_start_us":661,"threads_started":1}
I20260806 20:24:41.064796 17942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [term 1 LEADER]: Becoming Leader. State: Replica: 658a329c10f04411a67d849dbb16b301, State: Running, Role: LEADER
I20260806 20:24:41.066327 17942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658a329c10f04411a67d849dbb16b301" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34459 } }
I20260806 20:24:41.073082 17944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 658a329c10f04411a67d849dbb16b301. Latest consensus state: current_term: 1 leader_uuid: "658a329c10f04411a67d849dbb16b301" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658a329c10f04411a67d849dbb16b301" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34459 } } }
I20260806 20:24:41.073657 17944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [sys.catalog]: This master's current role is: LEADER
I20260806 20:24:41.074432 17943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "658a329c10f04411a67d849dbb16b301" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "658a329c10f04411a67d849dbb16b301" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34459 } } }
I20260806 20:24:41.074808 17943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301 [sys.catalog]: This master's current role is: LEADER
W20260806 20:24:41.081979 17958 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 20:24:41.082345 17958 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 20:24:41.082593 17959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:24:41.084770 17959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:24:41.100885 17959 catalog_manager.cc:1347] Generated new cluster ID: 6d6c80d7410e4491bb5c2e8fd001f5d4
I20260806 20:24:41.101078 17959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:24:41.134155 17959 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:24:41.135926 17959 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:24:41.144071 17959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 658a329c10f04411a67d849dbb16b301: Generated new TSK 0
I20260806 20:24:41.145450 17959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:24:41.203557 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34459
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:41.548149 17965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:41.548475 17965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:41.548640 17965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:41.556082 17965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:41.556231 17965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:41.556397 17965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:24:41.567035 17965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:41.568450 17965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:41.569799 17965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:41.585057 17971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:41.585215 17974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:41.585256 17972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:41.586591 17965 server_base.cc:1034] running on GCE node
I20260806 20:24:41.587350 17965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:41.588436 17965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:41.589679 17965 hybrid_clock.cc:648] HybridClock initialized: now 1786047881589624 us; error 51 us; skew 500 ppm
I20260806 20:24:41.590057 17965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:41 dist-test-slave-6k22 krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047881, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:41.593439 17965 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:24:41.595064 17965 webserver.cc:460] Webserver started at http://127.17.109.193:40427/ using document root <none> and password file <none>
I20260806 20:24:41.595834 17965 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:41.595968 17965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:41.596338 17965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:41.599404 17965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "d163e1a42a1c489d8f04bb30631bb5ef"
format_stamp: "Formatted at 2026-08-06 20:24:41 on dist-test-slave-6k22"
server_key: "2fbb3cdf4a5026b1f1e8af5d22fb56d0"
server_key_iv: "18633a071e9394babe6d92100f81ab91"
server_key_version: "encryptionkey@0"
I20260806 20:24:41.600333 17965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "d163e1a42a1c489d8f04bb30631bb5ef"
format_stamp: "Formatted at 2026-08-06 20:24:41 on dist-test-slave-6k22"
server_key: "2fbb3cdf4a5026b1f1e8af5d22fb56d0"
server_key_iv: "18633a071e9394babe6d92100f81ab91"
server_key_version: "encryptionkey@0"
I20260806 20:24:41.605885 17965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:24:41.609717 17981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:41.611112 17965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:24:41.611311 17965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "d163e1a42a1c489d8f04bb30631bb5ef"
format_stamp: "Formatted at 2026-08-06 20:24:41 on dist-test-slave-6k22"
server_key: "2fbb3cdf4a5026b1f1e8af5d22fb56d0"
server_key_iv: "18633a071e9394babe6d92100f81ab91"
server_key_version: "encryptionkey@0"
I20260806 20:24:41.611512 17965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:41.631120 17965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:41.635389 17965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:41.635741 17965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:41.636888 17965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:24:41.638584 17965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:41.638687 17965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:41.638892 17965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:41.638969 17965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:41.680970 17965 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:43251
I20260806 20:24:41.681058 18099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:43251 every 8 connection(s)
I20260806 20:24:41.682324 17965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:24:41.685241 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 17965
I20260806 20:24:41.685549 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:24:41.686482 17847 external_mini_cluster.cc:1428] Setting key 059116f5607a0c9bdbc2857708d17cfa
Aug 06 20:24:41 dist-test-slave-6k22 krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047881, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
I20260806 20:24:41.711246 18102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:41.686161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:46081 (local address 127.17.109.254:34459)
0806 20:24:41.686835 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:41.686842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:41.687999 (+ 1157us) server_negotiation.cc:407] Connection header received
0806 20:24:41.691089 (+ 3090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:41.691093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:41.691178 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:41.691299 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:41.694421 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:41.695599 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:41.698346 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:41.698909 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:41.699001 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:41.703348 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:41.703359 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:41.703368 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:41.703419 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:41.707647 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:41.708495 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:41.708501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:41.708504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:41.708698 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:41.709231 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:41.709237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:41.709241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:41.709705 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:41.709898 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:41.710768 (+ 870us) server_negotiation.cc:299] Negotiation successful
0806 20:24:41.710949 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":310,"threads_started":1}
I20260806 20:24:41.712440 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:41.686800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34459 (local address 127.17.109.193:46081)
0806 20:24:41.687792 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:41.687853 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:41.690692 (+ 2839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:41.691678 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:41.691696 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:41.693001 (+ 1305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:41.694172 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:41.694208 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:41.695793 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:41.695804 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:41.698094 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:41.698124 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:41.699357 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:41.699362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:41.699666 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:41.700475 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:41.700510 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:41.703076 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:41.707794 (+ 4718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:41.707802 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:41.707816 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:41.708338 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:41.708861 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:41.708865 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:41.708868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:41.709079 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:41.709872 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:41.709879 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:41.710237 (+ 358us) client_negotiation.cc:769] Sending connection context
0806 20:24:41.710629 (+ 392us) client_negotiation.cc:240] Negotiation successful
0806 20:24:41.711104 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":418,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
I20260806 20:24:41.714443 18100 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34459
I20260806 20:24:41.714947 18100 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:41.716187 18100 heartbeater.cc:511] Master 127.17.109.254:34459 requested a full tablet report, sending...
I20260806 20:24:41.720176 17901 ts_manager.cc:194] Registered new tserver with Master: d163e1a42a1c489d8f04bb30631bb5ef (127.17.109.193:43251)
I20260806 20:24:41.722369 17901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:46081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:24:41.742672 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34459
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:42.210727 18106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:42.211024 18106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:42.211215 18106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:42.219805 18106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:42.219939 18106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:42.220140 18106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:24:42.231058 18106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:42.232527 18106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:42.233954 18106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:42.249116 18111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:42.249188 18113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:42.249090 18115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:42.250626 18106 server_base.cc:1034] running on GCE node
I20260806 20:24:42.251240 18106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:42.252266 18106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:42.253499 18106 hybrid_clock.cc:648] HybridClock initialized: now 1786047882253455 us; error 41 us; skew 500 ppm
I20260806 20:24:42.253821 18106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:42 dist-test-slave-6k22 krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047882, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:42.257519 18106 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:24:42.259057 18106 webserver.cc:460] Webserver started at http://127.17.109.194:35555/ using document root <none> and password file <none>
I20260806 20:24:42.259867 18106 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:42.260018 18106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:42.260365 18106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:42.264045 18106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "f48ee38c176548f1b505b28a2a3aa3d5"
format_stamp: "Formatted at 2026-08-06 20:24:42 on dist-test-slave-6k22"
server_key: "79025fbeb80305d0c643f1a40e1bc6d9"
server_key_iv: "2f876dd36e597a65f32c18cb956b5636"
server_key_version: "encryptionkey@0"
I20260806 20:24:42.265005 18106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "f48ee38c176548f1b505b28a2a3aa3d5"
format_stamp: "Formatted at 2026-08-06 20:24:42 on dist-test-slave-6k22"
server_key: "79025fbeb80305d0c643f1a40e1bc6d9"
server_key_iv: "2f876dd36e597a65f32c18cb956b5636"
server_key_version: "encryptionkey@0"
I20260806 20:24:42.270900 18106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:24:42.274850 18122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:42.276333 18106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:24:42.276507 18106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "f48ee38c176548f1b505b28a2a3aa3d5"
format_stamp: "Formatted at 2026-08-06 20:24:42 on dist-test-slave-6k22"
server_key: "79025fbeb80305d0c643f1a40e1bc6d9"
server_key_iv: "2f876dd36e597a65f32c18cb956b5636"
server_key_version: "encryptionkey@0"
I20260806 20:24:42.276703 18106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:42.313737 18106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:42.319151 18106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:42.319490 18106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:42.320631 18106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:24:42.322124 18106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:42.322223 18106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:42.322407 18106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:42.322464 18106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:42.362120 18106 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:41995
I20260806 20:24:42.362213 18238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:41995 every 8 connection(s)
I20260806 20:24:42.363497 18106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:24:42.366894 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 18106
I20260806 20:24:42.367173 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:24:42.367764 17847 external_mini_cluster.cc:1428] Setting key 5328759492292ffaec69db8e2431ecf3
Aug 06 20:24:42 dist-test-slave-6k22 krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047882, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:42.389074 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:42.366440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:38345 (local address 127.17.109.254:34459)
0806 20:24:42.367089 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:42.367097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:42.370298 (+ 3201us) server_negotiation.cc:407] Connection header received
0806 20:24:42.370353 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:42.370356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:42.370418 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:42.370503 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:42.373396 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.374148 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.376538 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.376999 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.377140 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:42.382130 (+ 4990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:42.382136 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:42.382140 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:42.382184 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:42.385236 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:42.386286 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:42.386290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:42.386293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:42.386445 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:42.387042 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:42.387046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:42.387048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:42.387382 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:42.387505 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:42.388574 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0806 20:24:42.388790 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":360,"threads_started":1}
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
I20260806 20:24:42.390700 18242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:42.367199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34459 (local address 127.17.109.194:38345)
0806 20:24:42.368205 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:42.368261 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:42.370008 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:42.370857 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:42.370873 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:42.372059 (+ 1186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:42.373104 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:42.373139 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.374347 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.374387 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:42.376379 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:42.376392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.377238 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.377246 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:42.377499 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:42.378268 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:42.378304 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:42.381844 (+ 3540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:42.385395 (+ 3551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:42.385408 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:42.385427 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:42.386120 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:42.386573 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:42.386581 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:42.386587 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:42.386898 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:42.387523 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:42.387531 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:42.388087 (+ 556us) client_negotiation.cc:769] Sending connection context
0806 20:24:42.388436 (+ 349us) client_negotiation.cc:240] Negotiation successful
0806 20:24:42.389067 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":493,"threads_started":1}
I20260806 20:24:42.392705 18239 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34459
I20260806 20:24:42.393096 18239 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
I20260806 20:24:42.394130 18239 heartbeater.cc:511] Master 127.17.109.254:34459 requested a full tablet report, sending...
I20260806 20:24:42.395782 17901 ts_manager.cc:194] Registered new tserver with Master: f48ee38c176548f1b505b28a2a3aa3d5 (127.17.109.194:41995)
I20260806 20:24:42.396708 17901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:38345
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:24:42.419862 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34459
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:35313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:24:42.726140 18100 heartbeater.cc:503] Master 127.17.109.254:34459 was elected leader, sending a full tablet report...
W20260806 20:24:42.763048 18245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:42.763374 18245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:42.763599 18245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:42.771522 18245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:42.771704 18245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:42.771906 18245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:24:42.782802 18245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:35313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:42.784245 18245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:42.785691 18245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:42.800230 18250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:42.800297 18251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:42.800330 18253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:42.801275 18245 server_base.cc:1034] running on GCE node
I20260806 20:24:42.801994 18245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:42.802976 18245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:42.804193 18245 hybrid_clock.cc:648] HybridClock initialized: now 1786047882804046 us; error 131 us; skew 500 ppm
I20260806 20:24:42.804478 18245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:42 dist-test-slave-6k22 krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047882, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:42.807749 18245 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:24:42.809166 18245 webserver.cc:460] Webserver started at http://127.17.109.195:33305/ using document root <none> and password file <none>
I20260806 20:24:42.809831 18245 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:42.809944 18245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:42.810258 18245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:42.813400 18245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "f8620f4f7e8c43338e746988cccc50de"
format_stamp: "Formatted at 2026-08-06 20:24:42 on dist-test-slave-6k22"
server_key: "d79240a3168c07cb9e701b35be17664e"
server_key_iv: "4d4ba4e65f27c404fd02ba84e0b1667d"
server_key_version: "encryptionkey@0"
I20260806 20:24:42.814225 18245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "f8620f4f7e8c43338e746988cccc50de"
format_stamp: "Formatted at 2026-08-06 20:24:42 on dist-test-slave-6k22"
server_key: "d79240a3168c07cb9e701b35be17664e"
server_key_iv: "4d4ba4e65f27c404fd02ba84e0b1667d"
server_key_version: "encryptionkey@0"
I20260806 20:24:42.820195 18245 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 20:24:42.823784 18261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:42.825146 18245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:24:42.825343 18245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "f8620f4f7e8c43338e746988cccc50de"
format_stamp: "Formatted at 2026-08-06 20:24:42 on dist-test-slave-6k22"
server_key: "d79240a3168c07cb9e701b35be17664e"
server_key_iv: "4d4ba4e65f27c404fd02ba84e0b1667d"
server_key_version: "encryptionkey@0"
I20260806 20:24:42.825548 18245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:42.841794 18245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:42.846010 18245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:42.846292 18245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:42.847231 18245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:24:42.848650 18245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:42.848763 18245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:42.848875 18245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:42.848920 18245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:42.884918 18245 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:38609
I20260806 20:24:42.884970 18376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:38609 every 8 connection(s)
I20260806 20:24:42.886217 18245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:24:42.889382 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 18245
I20260806 20:24:42.889576 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:24:42.890170 17847 external_mini_cluster.cc:1428] Setting key fdb86a893ca62de1b45a311f943d4c64
Aug 06 20:24:42 dist-test-slave-6k22 krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047882, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:42.907625 18241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:42.889183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:60751 (local address 127.17.109.254:34459)
0806 20:24:42.889399 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:42.889403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:42.891003 (+ 1600us) server_negotiation.cc:407] Connection header received
0806 20:24:42.893530 (+ 2527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:42.893534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:42.893600 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:42.893681 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:42.895531 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.896316 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.897893 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.898274 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.898385 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:42.901996 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:42.902001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:42.902004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:42.902040 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:42.904538 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:42.905210 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:42.905218 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:42.905221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:42.905350 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:42.905762 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:42.905765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:42.905767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:42.906135 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:42.906253 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:42.907079 (+ 826us) server_negotiation.cc:299] Negotiation successful
0806 20:24:42.907328 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 20:24:42.908581 18379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:42.889733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34459 (local address 127.17.109.195:60751)
0806 20:24:42.890682 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:42.890741 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:42.893163 (+ 2422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:42.894065 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:42.894079 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:42.894744 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:42.895367 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:42.895387 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.896491 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.896496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:42.897769 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:42.897781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.898543 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.898549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:42.898880 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:42.899549 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:42.899637 (+ 88us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:42.901803 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:42.904654 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:42.904660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:42.904672 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:42.905081 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:42.905460 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:42.905466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:42.905470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:42.905655 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:42.906262 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:42.906271 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:42.906621 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 20:24:42.906957 (+ 336us) client_negotiation.cc:240] Negotiation successful
0806 20:24:42.907425 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":376,"threads_started":1}
I20260806 20:24:42.910109 18377 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34459
I20260806 20:24:42.910404 18377 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:42.911121 18377 heartbeater.cc:511] Master 127.17.109.254:34459 requested a full tablet report, sending...
I20260806 20:24:42.912585 17901 ts_manager.cc:194] Registered new tserver with Master: f8620f4f7e8c43338e746988cccc50de (127.17.109.195:38609)
I20260806 20:24:42.913410 17901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:60751
I20260806 20:24:42.919461 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 20:24:42 dist-test-slave-6k22 krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047882, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 20:24:42 dist-test-slave-6k22 krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047882, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:24:42.974743 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:42.959480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32970 (local address 127.17.109.193:43251)
0806 20:24:42.960241 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:42.960249 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:42.960278 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:24:42.960340 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:42.960347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:42.960559 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:42.960692 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:42.962136 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.963230 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.964530 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:42.965086 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:42.965184 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:42.969205 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:42.969215 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:42.969229 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:42.969262 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:42.971995 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:42.972755 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:42.972761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:42.972763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:42.972912 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:42.973317 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:42.973321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:42.973322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:42.973640 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:42.973887 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:42.974157 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 20:24:42.974352 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":454,"threads_started":1}
W20260806 20:24:42.976178 18010 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32970
Aug 06 20:24:43 dist-test-slave-6k22 krb5kdc[17854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 20:24:43 dist-test-slave-6k22 krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:24:43.027458 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:43.013108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.17.109.193:43251)
0806 20:24:43.013405 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:43.013412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:43.013598 (+ 186us) server_negotiation.cc:407] Connection header received
0806 20:24:43.013811 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:43.013814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:43.013934 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:43.014023 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:43.015452 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:43.016297 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:43.017734 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:43.018253 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:43.018344 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:43.022352 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:43.022356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:43.022360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:43.022395 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:43.025218 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:43.025831 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:43.025836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:43.025839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:43.025978 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:43.026315 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:43.026320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:43.026322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:43.026576 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:43.026780 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:43.027033 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 20:24:43.027208 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260806 20:24:43.029516 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 17965
I20260806 20:24:43.043368 18384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:43.030748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.17.109.193:43251)
0806 20:24:43.030994 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:43.031001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:43.031247 (+ 246us) server_negotiation.cc:407] Connection header received
0806 20:24:43.031337 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:43.031341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:43.031396 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:43.031479 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:43.032667 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:43.033358 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:43.034654 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:43.035101 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:43.035198 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:43.037508 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:43.037514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:43.037518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:43.037553 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:43.040648 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:43.041344 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:43.041350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:43.041353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:43.041510 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:43.041842 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:43.041845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:43.041855 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:43.042123 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:43.042333 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:43.042717 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 20:24:43.042916 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 20:24:43.044411 18095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:43.136850 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 17965
I20260806 20:24:43.156356 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 18106
Aug 06 20:24:43 dist-test-slave-6k22 krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:24:43.173133 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:43.157302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.17.109.194:41995)
0806 20:24:43.157965 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:43.157972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:43.157998 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:24:43.158060 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:43.158066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:43.158279 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:43.158420 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:43.159711 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:43.160810 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:43.162049 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:43.162727 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:43.162898 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:43.167152 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:43.167163 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:43.167176 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:43.167217 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:43.170267 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:43.170876 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:43.170883 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:43.170885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:43.171004 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:43.171402 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:43.171406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:43.171408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:43.171956 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:43.172193 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:43.172594 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 20:24:43.172821 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":408,"threads_started":1}
I20260806 20:24:43.174417 18234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:43.258049 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 18106
I20260806 20:24:43.277643 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 18245
Aug 06 20:24:43 dist-test-slave-6k22 krb5kdc[17854](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:24:43.295735 18393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:43.278925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44902 (local address 127.17.109.195:38609)
0806 20:24:43.279638 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:43.279644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:43.279676 (+ 32us) server_negotiation.cc:407] Connection header received
0806 20:24:43.279739 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:43.279746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:43.279968 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:43.280101 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:43.281782 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:43.282869 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:43.284651 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:43.285277 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:43.285378 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:43.289706 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:43.289718 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:43.289741 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:43.289780 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:43.292889 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:43.293584 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:43.293591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:43.293593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:43.293771 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:43.294173 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:43.294176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:43.294178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:43.294510 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:43.294793 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:43.295020 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 20:24:43.295221 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":344,"threads_started":1}
I20260806 20:24:43.297019 18372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:43.381243 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 18245
I20260806 20:24:43.400969 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 17868
I20260806 20:24:43.402262 17932 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:43.509279 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 17868
2026-08-06T20:24:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4493 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:24:43 dist-test-slave-6k22 krb5kdc[18398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:24:43 dist-test-slave-6k22 krb5kdc[18398](info): set up 2 sockets
Aug 06 20:24:43 dist-test-slave-6k22 krb5kdc[18398](info): commencing operation
krb5kdc: starting...
W20260806 20:24:44.676556 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:24:44 dist-test-slave-6k22 krb5kdc[18398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:24:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:24:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:24:44.886726 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46273
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:45.219235 18412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:45.219537 18412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:45.219713 18412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:45.227874 18412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:24:45.227996 18412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:45.228050 18412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:45.228116 18412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:24:45.228170 18412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:24:45.238400 18412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:45.239856 18412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:45.241204 18412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:45.253501 18418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:45.253468 18420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:45.253691 18417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:45.254317 18412 server_base.cc:1034] running on GCE node
I20260806 20:24:45.255216 18412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:45.256700 18412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:45.257928 18412 hybrid_clock.cc:648] HybridClock initialized: now 1786047885257868 us; error 48 us; skew 500 ppm
I20260806 20:24:45.258296 18412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:45 dist-test-slave-6k22 krb5kdc[18398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:45.267943 18412 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:24:45.269637 18412 webserver.cc:460] Webserver started at http://127.17.109.254:40363/ using document root <none> and password file <none>
I20260806 20:24:45.270378 18412 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:45.270535 18412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:45.270877 18412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:45.274192 18412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "c889bd84670c46bcb7cb08e0f78be828"
format_stamp: "Formatted at 2026-08-06 20:24:45 on dist-test-slave-6k22"
server_key: "352c3a45acd2df88b4b9b1894001fffe"
server_key_iv: "0f0ade4959420037ab6feca6dab53ffc"
server_key_version: "encryptionkey@0"
I20260806 20:24:45.275027 18412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "c889bd84670c46bcb7cb08e0f78be828"
format_stamp: "Formatted at 2026-08-06 20:24:45 on dist-test-slave-6k22"
server_key: "352c3a45acd2df88b4b9b1894001fffe"
server_key_iv: "0f0ade4959420037ab6feca6dab53ffc"
server_key_version: "encryptionkey@0"
I20260806 20:24:45.280797 18412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 20:24:45.284582 18427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:45.286229 18412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:24:45.286441 18412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "c889bd84670c46bcb7cb08e0f78be828"
format_stamp: "Formatted at 2026-08-06 20:24:45 on dist-test-slave-6k22"
server_key: "352c3a45acd2df88b4b9b1894001fffe"
server_key_iv: "0f0ade4959420037ab6feca6dab53ffc"
server_key_version: "encryptionkey@0"
I20260806 20:24:45.286665 18412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:45.316829 18412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:45.321807 18412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:45.322129 18412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:45.343083 18412 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:46273
I20260806 20:24:45.343106 18485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:46273 every 8 connection(s)
I20260806 20:24:45.344681 18412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:24:45.346385 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 18412
I20260806 20:24:45.346632 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:24:45.347447 17847 external_mini_cluster.cc:1428] Setting key 1f06106f86f8f5a29e939ba36a2bd5d4
I20260806 20:24:45.349386 18486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:45.361058 18486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828: Bootstrap starting.
Aug 06 20:24:45 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:45.364964 18486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:45.366097 18486 log.cc:824] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:45.369568 18486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828: No bootstrap required, opened a new log
I20260806 20:24:45.371088 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:45.349689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.17.109.254:46273)
0806 20:24:45.350501 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:45.350513 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:45.350554 (+ 41us) server_negotiation.cc:407] Connection header received
0806 20:24:45.352381 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:45.352401 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:45.352795 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:45.353250 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:45.354832 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:45.356368 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:45.357729 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:45.358502 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:45.358759 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:45.363404 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:45.363412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:45.363427 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:45.363463 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:45.366864 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:45.367584 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:45.367594 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:45.367607 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:45.367771 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:45.368196 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:45.368199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:45.368201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:45.368737 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:45.368954 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:45.369561 (+ 607us) server_negotiation.cc:299] Negotiation successful
0806 20:24:45.370109 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":364,"threads_started":1}
I20260806 20:24:45.377837 18486 raft_consensus.cc:348] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c889bd84670c46bcb7cb08e0f78be828" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46273 } }
I20260806 20:24:45.378197 18486 raft_consensus.cc:374] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:45.378335 18486 raft_consensus.cc:729] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c889bd84670c46bcb7cb08e0f78be828, State: Initialized, Role: FOLLOWER
I20260806 20:24:45.378881 18486 consensus_queue.cc:260] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c889bd84670c46bcb7cb08e0f78be828" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46273 } }
I20260806 20:24:45.379103 18486 raft_consensus.cc:388] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:24:45.379242 18486 raft_consensus.cc:482] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:24:45.379415 18486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:45.381582 18486 raft_consensus.cc:504] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c889bd84670c46bcb7cb08e0f78be828" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46273 } }
I20260806 20:24:45.382081 18486 leader_election.cc:302] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c889bd84670c46bcb7cb08e0f78be828; no voters:
I20260806 20:24:45.382848 18486 leader_election.cc:288] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:24:45.382995 18491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:45.383769 18491 raft_consensus.cc:686] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [term 1 LEADER]: Becoming Leader. State: Replica: c889bd84670c46bcb7cb08e0f78be828, State: Running, Role: LEADER
I20260806 20:24:45.384310 18491 consensus_queue.cc:237] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c889bd84670c46bcb7cb08e0f78be828" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46273 } }
I20260806 20:24:45.384990 18486 sys_catalog.cc:564] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:24:45.387466 18493 sys_catalog.cc:455] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c889bd84670c46bcb7cb08e0f78be828. Latest consensus state: current_term: 1 leader_uuid: "c889bd84670c46bcb7cb08e0f78be828" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c889bd84670c46bcb7cb08e0f78be828" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46273 } } }
I20260806 20:24:45.387435 18492 sys_catalog.cc:455] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c889bd84670c46bcb7cb08e0f78be828" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c889bd84670c46bcb7cb08e0f78be828" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46273 } } }
I20260806 20:24:45.388914 18493 sys_catalog.cc:458] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [sys.catalog]: This master's current role is: LEADER
I20260806 20:24:45.388971 18492 sys_catalog.cc:458] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828 [sys.catalog]: This master's current role is: LEADER
I20260806 20:24:45.390949 18502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:24:45.396317 18502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:24:45.404829 18502 catalog_manager.cc:1347] Generated new cluster ID: 0514b3811b9a4ca4ad28c2e145f94346
I20260806 20:24:45.404942 18502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:24:45.420568 18502 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:24:45.422501 18502 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:24:45.436105 18502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c889bd84670c46bcb7cb08e0f78be828: Generated new TSK 0
I20260806 20:24:45.437004 18502 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:24:45.499754 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46273
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:45.847397 18513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:45.847745 18513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:45.847926 18513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:45.854998 18513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:45.855098 18513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:45.855276 18513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:24:45.865154 18513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:45.866444 18513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:45.867844 18513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:45.880935 18522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:45.880901 18519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:45.881130 18518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:45.882040 18513 server_base.cc:1034] running on GCE node
I20260806 20:24:45.882602 18513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:45.883504 18513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:45.884773 18513 hybrid_clock.cc:648] HybridClock initialized: now 1786047885884710 us; error 51 us; skew 500 ppm
I20260806 20:24:45.885187 18513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:45 dist-test-slave-6k22 krb5kdc[18398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:45.888680 18513 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:24:45.890321 18513 webserver.cc:460] Webserver started at http://127.17.109.193:40731/ using document root <none> and password file <none>
I20260806 20:24:45.891160 18513 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:45.891310 18513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:45.891762 18513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:45.894892 18513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "f2332b06f9d846c284e6be4949829d99"
format_stamp: "Formatted at 2026-08-06 20:24:45 on dist-test-slave-6k22"
server_key: "6807beca898e6ef512580c745d8b71d2"
server_key_iv: "ee8325db4e36083c628b86c107fefb36"
server_key_version: "encryptionkey@0"
I20260806 20:24:45.895797 18513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "f2332b06f9d846c284e6be4949829d99"
format_stamp: "Formatted at 2026-08-06 20:24:45 on dist-test-slave-6k22"
server_key: "6807beca898e6ef512580c745d8b71d2"
server_key_iv: "ee8325db4e36083c628b86c107fefb36"
server_key_version: "encryptionkey@0"
I20260806 20:24:45.901427 18513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 20:24:45.905244 18529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:45.906621 18513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:24:45.906798 18513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "f2332b06f9d846c284e6be4949829d99"
format_stamp: "Formatted at 2026-08-06 20:24:45 on dist-test-slave-6k22"
server_key: "6807beca898e6ef512580c745d8b71d2"
server_key_iv: "ee8325db4e36083c628b86c107fefb36"
server_key_version: "encryptionkey@0"
I20260806 20:24:45.907029 18513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:45.935323 18513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:45.940227 18513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:45.940594 18513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:45.941756 18513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:24:45.943404 18513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:45.943495 18513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:45.943701 18513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:45.943794 18513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:45.980890 18513 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:35975
I20260806 20:24:45.980983 18649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:35975 every 8 connection(s)
I20260806 20:24:45.982538 18513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:24:45.983855 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 18513
I20260806 20:24:45.984148 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:24:45.984915 17847 external_mini_cluster.cc:1428] Setting key 422d94e0a3a444df3872265e77a15bf8
Aug 06 20:24:45 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047885, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
I20260806 20:24:46.008916 18653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:45.985983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:52815 (local address 127.17.109.254:46273)
0806 20:24:45.986760 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:45.986766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:45.987915 (+ 1149us) server_negotiation.cc:407] Connection header received
0806 20:24:45.990243 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:45.990253 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:45.990360 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:45.990497 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:45.992539 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:45.993891 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:45.996951 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:45.997401 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:45.997523 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:46.002419 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:46.002424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:46.002428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:46.002465 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:46.005429 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:46.006237 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:46.006248 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:46.006251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:46.006378 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:46.006859 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:46.006869 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:46.006872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:46.007151 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:46.007324 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:46.008204 (+ 880us) server_negotiation.cc:299] Negotiation successful
0806 20:24:46.008511 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":443,"threads_started":1}
I20260806 20:24:46.010159 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:45.986705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46273 (local address 127.17.109.193:52815)
0806 20:24:45.987704 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:45.987758 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:45.989888 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:45.990907 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:45.990917 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:45.991710 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:45.992350 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:45.992371 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:45.994118 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:45.994126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:45.996785 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:45.996806 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:45.997692 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:45.997703 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:45.998109 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:45.998972 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:45.999023 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:46.002140 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:46.005588 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:46.005599 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:46.005617 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:46.006091 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:46.006509 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:46.006515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:46.006519 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:46.006731 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:46.007301 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:46.007309 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:46.007786 (+ 477us) client_negotiation.cc:769] Sending connection context
0806 20:24:46.008075 (+ 289us) client_negotiation.cc:240] Negotiation successful
0806 20:24:46.008684 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":414,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
I20260806 20:24:46.012244 18650 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46273
I20260806 20:24:46.012590 18650 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:46.013381 18650 heartbeater.cc:511] Master 127.17.109.254:46273 requested a full tablet report, sending...
I20260806 20:24:46.015676 18445 ts_manager.cc:194] Registered new tserver with Master: f2332b06f9d846c284e6be4949829d99 (127.17.109.193:35975)
I20260806 20:24:46.017271 18445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:52815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:24:46.034682 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46273
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:46.362042 18657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:46.362310 18657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:46.362469 18657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:46.369664 18657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:46.369773 18657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:46.369975 18657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:24:46.379798 18657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:46.381108 18657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:46.382514 18657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:46.396137 18665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:46.396174 18663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:46.396226 18662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:46.397737 18657 server_base.cc:1034] running on GCE node
I20260806 20:24:46.398324 18657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:46.399281 18657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:46.400472 18657 hybrid_clock.cc:648] HybridClock initialized: now 1786047886400424 us; error 40 us; skew 500 ppm
I20260806 20:24:46.400791 18657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:46 dist-test-slave-6k22 krb5kdc[18398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:46.404188 18657 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:24:46.405794 18657 webserver.cc:460] Webserver started at http://127.17.109.194:42821/ using document root <none> and password file <none>
I20260806 20:24:46.406498 18657 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:46.406647 18657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:46.406978 18657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:46.409992 18657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "eb48de61e549445582338e094e1859ba"
format_stamp: "Formatted at 2026-08-06 20:24:46 on dist-test-slave-6k22"
server_key: "8c94009e79672b82adbd9f68d6acef0c"
server_key_iv: "52090b50ce3a98f1acdd0117380b1912"
server_key_version: "encryptionkey@0"
I20260806 20:24:46.410900 18657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "eb48de61e549445582338e094e1859ba"
format_stamp: "Formatted at 2026-08-06 20:24:46 on dist-test-slave-6k22"
server_key: "8c94009e79672b82adbd9f68d6acef0c"
server_key_iv: "52090b50ce3a98f1acdd0117380b1912"
server_key_version: "encryptionkey@0"
I20260806 20:24:46.416299 18657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 20:24:46.419935 18672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:46.421206 18657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:24:46.421370 18657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "eb48de61e549445582338e094e1859ba"
format_stamp: "Formatted at 2026-08-06 20:24:46 on dist-test-slave-6k22"
server_key: "8c94009e79672b82adbd9f68d6acef0c"
server_key_iv: "52090b50ce3a98f1acdd0117380b1912"
server_key_version: "encryptionkey@0"
I20260806 20:24:46.421564 18657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:46.442245 18657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:46.446432 18657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:46.446732 18657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:46.447746 18657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:24:46.449120 18657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:46.449198 18657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:46.449362 18657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:46.449415 18657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:46.488719 18657 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:40403
I20260806 20:24:46.488826 18789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:40403 every 8 connection(s)
I20260806 20:24:46.490361 18657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:24:46.494987 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 18657
I20260806 20:24:46.495187 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:24:46.495678 17847 external_mini_cluster.cc:1428] Setting key a6be2ab4534d01a88797b542fc86c526
Aug 06 20:24:46 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:46.512720 18653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:46.493290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:56589 (local address 127.17.109.254:46273)
0806 20:24:46.493477 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:46.493482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:46.494816 (+ 1334us) server_negotiation.cc:407] Connection header received
0806 20:24:46.497227 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:46.497234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:46.497337 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:46.497483 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:46.499208 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:46.500138 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:46.501918 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:46.502452 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:46.502634 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:46.506295 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:46.506300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:46.506304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:46.506337 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:46.509510 (+ 3173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:46.510233 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:46.510238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:46.510242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:46.510379 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:46.510864 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:46.510870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:46.510874 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:46.511156 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:46.511309 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:46.512122 (+ 813us) server_negotiation.cc:299] Negotiation successful
0806 20:24:46.512368 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:24:46.513737 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:46.493782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46273 (local address 127.17.109.194:56589)
0806 20:24:46.494605 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:46.494666 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:46.496897 (+ 2231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:46.497765 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:46.497774 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:46.498364 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:46.499003 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:46.499023 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:46.500325 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:46.500332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:46.501741 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:46.501755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:46.502730 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:46.502739 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:46.503028 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:46.503842 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:46.503870 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:46.506062 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:46.509651 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:46.509660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:46.509678 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:46.510105 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:46.510488 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:46.510493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:46.510496 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:46.510752 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:46.511284 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:46.511292 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:46.511737 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 20:24:46.512034 (+ 297us) client_negotiation.cc:240] Negotiation successful
0806 20:24:46.512580 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":306,"threads_started":1}
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
I20260806 20:24:46.515237 18790 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46273
I20260806 20:24:46.515537 18790 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:46.516331 18790 heartbeater.cc:511] Master 127.17.109.254:46273 requested a full tablet report, sending...
I20260806 20:24:46.517808 18445 ts_manager.cc:194] Registered new tserver with Master: eb48de61e549445582338e094e1859ba (127.17.109.194:40403)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
I20260806 20:24:46.518817 18445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:56589
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:24:46.543808 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46273
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:46.875285 18796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:46.875598 18796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:46.875715 18796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:46.883116 18796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:46.883217 18796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:46.883332 18796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:24:46.893363 18796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:46.894572 18796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:46.895915 18796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:46.909906 18805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:46.909898 18801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:46.909898 18803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:46.911254 18796 server_base.cc:1034] running on GCE node
I20260806 20:24:46.911966 18796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:46.912923 18796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:46.914143 18796 hybrid_clock.cc:648] HybridClock initialized: now 1786047886914083 us; error 61 us; skew 500 ppm
I20260806 20:24:46.914454 18796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:46 dist-test-slave-6k22 krb5kdc[18398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:46.917820 18796 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:24:46.919401 18796 webserver.cc:460] Webserver started at http://127.17.109.195:38243/ using document root <none> and password file <none>
I20260806 20:24:46.920112 18796 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:46.920250 18796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:46.920567 18796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:46.923465 18796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "59aa39b90e21459187521a7ae1516d7e"
format_stamp: "Formatted at 2026-08-06 20:24:46 on dist-test-slave-6k22"
server_key: "1050feac50625ff747819352675a016f"
server_key_iv: "1006535fe88b3b5f2b5dcc78d03ed925"
server_key_version: "encryptionkey@0"
I20260806 20:24:46.924311 18796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "59aa39b90e21459187521a7ae1516d7e"
format_stamp: "Formatted at 2026-08-06 20:24:46 on dist-test-slave-6k22"
server_key: "1050feac50625ff747819352675a016f"
server_key_iv: "1006535fe88b3b5f2b5dcc78d03ed925"
server_key_version: "encryptionkey@0"
I20260806 20:24:46.929678 18796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:24:46.933188 18812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:46.934492 18796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:24:46.934662 18796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "59aa39b90e21459187521a7ae1516d7e"
format_stamp: "Formatted at 2026-08-06 20:24:46 on dist-test-slave-6k22"
server_key: "1050feac50625ff747819352675a016f"
server_key_iv: "1006535fe88b3b5f2b5dcc78d03ed925"
server_key_version: "encryptionkey@0"
I20260806 20:24:46.934862 18796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:46.953300 18796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:46.957785 18796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:46.958091 18796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:46.959110 18796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:24:46.960532 18796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:46.960632 18796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:46.960781 18796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:46.960851 18796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:46.996311 18796 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:36151
I20260806 20:24:46.996361 18932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:36151 every 8 connection(s)
I20260806 20:24:46.997624 18796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:24:47.004091 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 18796
I20260806 20:24:47.004313 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:24:47.004976 17847 external_mini_cluster.cc:1428] Setting key 3a7ad4867a4875dd6dabb9784d702b45
Aug 06 20:24:47 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047886, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:47.019743 18653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.000248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:45827 (local address 127.17.109.254:46273)
0806 20:24:47.000419 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.000423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.001501 (+ 1078us) server_negotiation.cc:407] Connection header received
0806 20:24:47.003289 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.003294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.003365 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.003453 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:47.005707 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.006809 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.008875 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.009285 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.009383 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.013539 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:47.013543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:47.013547 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:47.013580 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:47.016268 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.017113 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.017117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.017119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.017244 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.017781 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.017785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.017787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.018039 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:47.018147 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.019155 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.019329 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 20:24:47.020344 18650 heartbeater.cc:503] Master 127.17.109.254:46273 was elected leader, sending a full tablet report...
I20260806 20:24:47.021293 18935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.000632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46273 (local address 127.17.109.195:45827)
0806 20:24:47.001326 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:47.001371 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:47.003029 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:47.003759 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:47.003772 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:47.004617 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:47.005490 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.005518 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.006986 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.006992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.008744 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.008760 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.009558 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.009565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.009815 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.010566 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:47.010601 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:47.013338 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:47.016417 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:47.016426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:47.016459 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:47.016997 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:47.017407 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:47.017412 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:47.017414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:47.017621 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:47.018147 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:47.018153 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:47.018604 (+ 451us) client_negotiation.cc:769] Sending connection context
0806 20:24:47.019014 (+ 410us) client_negotiation.cc:240] Negotiation successful
0806 20:24:47.019619 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":274,"threads_started":1}
I20260806 20:24:47.022691 18933 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46273
I20260806 20:24:47.022998 18933 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:47.023809 18933 heartbeater.cc:511] Master 127.17.109.254:46273 requested a full tablet report, sending...
I20260806 20:24:47.025274 18445 ts_manager.cc:194] Registered new tserver with Master: 59aa39b90e21459187521a7ae1516d7e (127.17.109.195:36151)
I20260806 20:24:47.026105 18445 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:45827
I20260806 20:24:47.033639 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:24:47.055796 18653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.041580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59956 (local address 127.17.109.254:46273)
0806 20:24:47.041904 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.041913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.042309 (+ 396us) server_negotiation.cc:407] Connection header received
0806 20:24:47.042450 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.042454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.042522 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.042613 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:47.044265 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.045076 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.046351 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.046819 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.046939 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.049890 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:47.049896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:47.049900 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:47.049936 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:47.052936 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.053635 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.053639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.053642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.053828 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.054233 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.054237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.054239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.054587 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:47.054904 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.055137 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.055324 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 20:24:47.065016 18445 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:24:47.073284 18445 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:24:47.114331 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.093194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39066 (local address 127.17.109.193:35975)
0806 20:24:47.101610 (+ 8416us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.101639 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.103947 (+ 2308us) server_negotiation.cc:407] Connection header received
0806 20:24:47.104095 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.104101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.104320 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.104608 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:47.105882 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.107754 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.110903 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.112602 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.112862 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.113067 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.113709 (+ 642us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.113908 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8132,"thread_start_us":7907,"threads_started":1}
I20260806 20:24:47.114499 18945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.091788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:40403 (local address 127.0.0.1:48838)
0806 20:24:47.093175 (+ 1387us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:47.093228 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:47.093487 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:47.106337 (+ 12850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:47.106344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:47.106373 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:47.106795 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.106807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.108914 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.108927 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.110901 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.110918 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.113092 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.113100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.113429 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.113485 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 20:24:47.113635 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 20:24:47.113903 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1120,"mutex_wait_us":24,"thread_start_us":947,"threads_started":1}
I20260806 20:24:47.114900 18949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.094016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:36151 (local address 127.0.0.1:58736)
0806 20:24:47.103254 (+ 9238us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:47.103278 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:47.103420 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:47.104750 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:47.104763 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:47.104815 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:47.105420 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.105433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.107924 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.107930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.110688 (+ 2758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.110713 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.113574 (+ 2861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.113579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.113792 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.113846 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 20:24:47.113967 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 20:24:47.114128 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9071,"mutex_wait_us":26,"thread_start_us":9342,"threads_started":1}
I20260806 20:24:47.115173 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.091919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48838 (local address 127.17.109.194:40403)
0806 20:24:47.105762 (+ 13843us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.105775 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.105810 (+ 35us) server_negotiation.cc:407] Connection header received
0806 20:24:47.105927 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.105933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.106119 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.106348 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:47.106985 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.108737 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.111142 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.112820 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.113076 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.113392 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.114611 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.114796 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13609,"thread_start_us":13436,"threads_started":1}
I20260806 20:24:47.115226 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.094451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.17.109.195:36151)
0806 20:24:47.104020 (+ 9569us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.104030 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.104066 (+ 36us) server_negotiation.cc:407] Connection header received
0806 20:24:47.104176 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.104183 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.104435 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.105650 (+ 1215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:47.105927 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.107751 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.110901 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.112835 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.113114 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.113392 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.114661 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.114855 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9390,"thread_start_us":9224,"threads_started":1}
I20260806 20:24:47.115252 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.092332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:35975 (local address 127.0.0.1:39066)
0806 20:24:47.102929 (+ 10597us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:47.102967 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:47.103204 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:47.104917 (+ 1713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:47.104924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:47.104962 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:47.105641 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.105659 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.107924 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.107930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.110733 (+ 2803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.110744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.113334 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.113342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.113493 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.113522 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 20:24:47.113635 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 20:24:47.113903 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10352,"mutex_wait_us":195,"thread_start_us":10127,"threads_started":1}
I20260806 20:24:47.119513 18721 tablet_service.cc:1467] Processing CreateTablet for tablet 18dc649cfcda40b7a28434941c292bbc (DEFAULT_TABLE table=test-table [id=1a73bc1483264c4695b294bf6ca955c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:47.120793 18580 tablet_service.cc:1467] Processing CreateTablet for tablet 18dc649cfcda40b7a28434941c292bbc (DEFAULT_TABLE table=test-table [id=1a73bc1483264c4695b294bf6ca955c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:47.121429 18721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18dc649cfcda40b7a28434941c292bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:47.122437 18580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18dc649cfcda40b7a28434941c292bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:47.122022 18864 tablet_service.cc:1467] Processing CreateTablet for tablet 18dc649cfcda40b7a28434941c292bbc (DEFAULT_TABLE table=test-table [id=1a73bc1483264c4695b294bf6ca955c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:47.123538 18864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18dc649cfcda40b7a28434941c292bbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:47.134857 18953 tablet_bootstrap.cc:492] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99: Bootstrap starting.
I20260806 20:24:47.138173 18953 tablet_bootstrap.cc:654] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:47.139531 18953 log.cc:824] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:47.139460 18955 tablet_bootstrap.cc:492] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba: Bootstrap starting.
I20260806 20:24:47.141448 18956 tablet_bootstrap.cc:492] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e: Bootstrap starting.
I20260806 20:24:47.144953 18955 tablet_bootstrap.cc:654] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:47.145231 18956 tablet_bootstrap.cc:654] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:47.146278 18956 log.cc:824] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:47.146459 18955 log.cc:824] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:47.149237 18956 tablet_bootstrap.cc:492] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e: No bootstrap required, opened a new log
I20260806 20:24:47.149302 18953 tablet_bootstrap.cc:492] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99: No bootstrap required, opened a new log
I20260806 20:24:47.149534 18956 ts_tablet_manager.cc:1397] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 20:24:47.149659 18953 ts_tablet_manager.cc:1397] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260806 20:24:47.149943 18955 tablet_bootstrap.cc:492] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba: No bootstrap required, opened a new log
I20260806 20:24:47.150226 18955 ts_tablet_manager.cc:1397] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 20:24:47.157375 18956 raft_consensus.cc:348] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.157769 18956 raft_consensus.cc:374] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:47.157907 18956 raft_consensus.cc:729] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa39b90e21459187521a7ae1516d7e, State: Initialized, Role: FOLLOWER
I20260806 20:24:47.158543 18956 consensus_queue.cc:260] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.159830 18933 heartbeater.cc:503] Master 127.17.109.254:46273 was elected leader, sending a full tablet report...
I20260806 20:24:47.160534 18956 ts_tablet_manager.cc:1428] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260806 20:24:47.162061 18953 raft_consensus.cc:348] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.162060 18955 raft_consensus.cc:348] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.162536 18955 raft_consensus.cc:374] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:47.162623 18953 raft_consensus.cc:374] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:47.162699 18955 raft_consensus.cc:729] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb48de61e549445582338e094e1859ba, State: Initialized, Role: FOLLOWER
I20260806 20:24:47.162779 18953 raft_consensus.cc:729] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2332b06f9d846c284e6be4949829d99, State: Initialized, Role: FOLLOWER
I20260806 20:24:47.163472 18953 consensus_queue.cc:260] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.163499 18955 consensus_queue.cc:260] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.167945 18953 ts_tablet_manager.cc:1428] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260806 20:24:47.170202 18790 heartbeater.cc:503] Master 127.17.109.254:46273 was elected leader, sending a full tablet report...
I20260806 20:24:47.170967 18955 ts_tablet_manager.cc:1428] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba: Time spent starting tablet: real 0.021s user 0.010s sys 0.012s
W20260806 20:24:47.235373 18651 tablet.cc:2369] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:24:47.242961 18791 tablet.cc:2369] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:24:47.249325 18934 tablet.cc:2369] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:24:47.249935 18961 raft_consensus.cc:482] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:24:47.250257 18961 raft_consensus.cc:504] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.252602 18961 leader_election.cc:288] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2332b06f9d846c284e6be4949829d99 (127.17.109.193:35975), 59aa39b90e21459187521a7ae1516d7e (127.17.109.195:36151)
I20260806 20:24:47.259743 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.252923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:48591 (local address 127.17.109.193:35975)
0806 20:24:47.253205 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.253215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.254054 (+ 839us) server_negotiation.cc:407] Connection header received
0806 20:24:47.254239 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.254243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.254301 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.254396 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:47.255073 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.255841 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.257762 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.258693 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.258847 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.258915 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.259319 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.259445 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 20:24:47.259796 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.253293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:42385 (local address 127.17.109.195:36151)
0806 20:24:47.253518 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.253525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.253973 (+ 448us) server_negotiation.cc:407] Connection header received
0806 20:24:47.254221 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.254225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.254285 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.254376 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:47.255022 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.255848 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.257793 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.258865 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.259014 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.259098 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.259461 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.259605 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 20:24:47.259840 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.252975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:35975 (local address 127.17.109.194:48591)
0806 20:24:47.253959 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:47.253973 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:47.254080 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:47.254408 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:47.254412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:47.254468 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:47.254904 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.254914 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.255969 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.255974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.257611 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.257626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.258935 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.258945 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.259167 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.259190 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 20:24:47.259292 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 20:24:47.259418 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":35,"spinlock_wait_cycles":28032,"thread_start_us":453,"threads_started":1}
I20260806 20:24:47.260545 18964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.252952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:36151 (local address 127.17.109.194:42385)
0806 20:24:47.253841 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:47.253863 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:47.254057 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:47.254404 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:47.254408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:47.254433 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:47.254862 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.254875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.255971 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.255979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.257642 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:47.257654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.259098 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.259106 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:47.259270 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.259304 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 20:24:47.259997 (+ 693us) client_negotiation.cc:240] Negotiation successful
0806 20:24:47.260126 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":7,"thread_start_us":738,"threads_started":1}
I20260806 20:24:47.261036 18600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18dc649cfcda40b7a28434941c292bbc" candidate_uuid: "eb48de61e549445582338e094e1859ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2332b06f9d846c284e6be4949829d99" is_pre_election: true
I20260806 20:24:47.261677 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18dc649cfcda40b7a28434941c292bbc" candidate_uuid: "eb48de61e549445582338e094e1859ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa39b90e21459187521a7ae1516d7e" is_pre_election: true
I20260806 20:24:47.261984 18600 raft_consensus.cc:2454] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb48de61e549445582338e094e1859ba in term 0.
I20260806 20:24:47.262347 18886 raft_consensus.cc:2454] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb48de61e549445582338e094e1859ba in term 0.
I20260806 20:24:47.262907 18676 leader_election.cc:302] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb48de61e549445582338e094e1859ba, f2332b06f9d846c284e6be4949829d99; no voters:
I20260806 20:24:47.263299 18961 raft_consensus.cc:2781] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:24:47.263450 18961 raft_consensus.cc:482] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:24:47.263587 18961 raft_consensus.cc:3037] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:47.265828 18961 raft_consensus.cc:504] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.266408 18961 leader_election.cc:288] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [CANDIDATE]: Term 1 election: Requested vote from peers f2332b06f9d846c284e6be4949829d99 (127.17.109.193:35975), 59aa39b90e21459187521a7ae1516d7e (127.17.109.195:36151)
I20260806 20:24:47.266820 18600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18dc649cfcda40b7a28434941c292bbc" candidate_uuid: "eb48de61e549445582338e094e1859ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2332b06f9d846c284e6be4949829d99"
I20260806 20:24:47.266885 18886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18dc649cfcda40b7a28434941c292bbc" candidate_uuid: "eb48de61e549445582338e094e1859ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59aa39b90e21459187521a7ae1516d7e"
I20260806 20:24:47.267005 18600 raft_consensus.cc:3037] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:47.267135 18886 raft_consensus.cc:3037] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:47.269356 18600 raft_consensus.cc:2454] T 18dc649cfcda40b7a28434941c292bbc P f2332b06f9d846c284e6be4949829d99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb48de61e549445582338e094e1859ba in term 1.
I20260806 20:24:47.269467 18886 raft_consensus.cc:2454] T 18dc649cfcda40b7a28434941c292bbc P 59aa39b90e21459187521a7ae1516d7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb48de61e549445582338e094e1859ba in term 1.
I20260806 20:24:47.269773 18676 leader_election.cc:302] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb48de61e549445582338e094e1859ba, f2332b06f9d846c284e6be4949829d99; no voters:
I20260806 20:24:47.270069 18961 raft_consensus.cc:2781] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:47.270576 18961 raft_consensus.cc:686] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [term 1 LEADER]: Becoming Leader. State: Replica: eb48de61e549445582338e094e1859ba, State: Running, Role: LEADER
I20260806 20:24:47.271066 18961 consensus_queue.cc:237] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } }
I20260806 20:24:47.276189 18445 catalog_manager.cc:5495] T 18dc649cfcda40b7a28434941c292bbc P eb48de61e549445582338e094e1859ba reported cstate change: term changed from 0 to 1, leader changed from <none> to eb48de61e549445582338e094e1859ba (127.17.109.194). New cstate: current_term: 1 leader_uuid: "eb48de61e549445582338e094e1859ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb48de61e549445582338e094e1859ba" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2332b06f9d846c284e6be4949829d99" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59aa39b90e21459187521a7ae1516d7e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36151 } health_report { overall_health: UNKNOWN } } }
Aug 06 20:24:47 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:24:47.316042 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.298954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39072 (local address 127.17.109.193:35975)
0806 20:24:47.299203 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.299210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.299955 (+ 745us) server_negotiation.cc:407] Connection header received
0806 20:24:47.300156 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.300160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.300229 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.300314 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:47.302213 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.303012 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.304433 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.304859 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.304991 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.309237 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:47.309254 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:47.309271 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:47.309320 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:47.312743 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.313547 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.313556 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.313559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.313730 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.314180 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.314184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.314187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.314587 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:47.314952 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.315322 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.315610 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 20:24:47 dist-test-slave-6k22 krb5kdc[18398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047887, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 20:24:47 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047887, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:24:47.375073 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.359447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.17.109.193:35975)
0806 20:24:47.359730 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.359737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.360106 (+ 369us) server_negotiation.cc:407] Connection header received
0806 20:24:47.360312 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.360316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.360384 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.360464 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:47.362287 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.363225 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.364783 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.365233 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.365339 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.369462 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:47.369467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:47.369470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:47.369510 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:47.372264 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.372899 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.372904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.372906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.373046 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.373502 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.373507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.373510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.373871 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:47.374151 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.374379 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.374699 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 20:24:47 dist-test-slave-6k22 krb5kdc[18398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 20:24:47 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:24:47.427060 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.412692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39082 (local address 127.17.109.193:35975)
0806 20:24:47.412916 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.412922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.413097 (+ 175us) server_negotiation.cc:407] Connection header received
0806 20:24:47.413280 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.413284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.413351 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.413432 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:47.414902 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.416045 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.417283 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.417777 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.417874 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.421862 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:47.421866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:47.421870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:47.421906 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:47.424436 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.425090 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.425094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.425096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.425217 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.425560 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.425565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.425569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.425916 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:47.426184 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.426464 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.426693 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 20:24:47.429538 18559 tablet_service.cc:2908] Rejecting scan request for tablet 18dc649cfcda40b7a28434941c292bbc: Uninitialized: clean time has not yet been initialized
I20260806 20:24:47.433295 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 18513
I20260806 20:24:47.447674 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.434440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.17.109.193:35975)
0806 20:24:47.434780 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.434788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.435009 (+ 221us) server_negotiation.cc:407] Connection header received
0806 20:24:47.435147 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.435152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.435217 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.435308 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:47.436545 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.437367 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.438739 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.439216 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.439430 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.441907 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:47.441913 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:47.441917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:47.441951 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:47.444984 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.445611 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.445662 (+ 51us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.445665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.445806 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.446138 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.446142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.446145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.446486 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:47.446779 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.446991 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.447168 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 20:24:47.448364 18645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:47.551182 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 18513
I20260806 20:24:47.571496 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 18657
Aug 06 20:24:47 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:24:47.587694 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.572630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48850 (local address 127.17.109.194:40403)
0806 20:24:47.572933 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.572939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.572962 (+ 23us) server_negotiation.cc:407] Connection header received
0806 20:24:47.573096 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.573101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.573167 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.573284 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:47.574513 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.575246 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.576619 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.577079 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.577224 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.581672 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:47.581689 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:47.581706 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:47.581762 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:47.584806 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.585516 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.585523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.585525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.585691 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.586103 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.586109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.586112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.586460 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:47.586795 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.587058 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.587305 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260806 20:24:47.588464 18785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:47.684036 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 18657
I20260806 20:24:47.704840 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 18796
Aug 06 20:24:47 dist-test-slave-6k22 krb5kdc[18398](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:24:47.721235 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:47.705723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58738 (local address 127.17.109.195:36151)
0806 20:24:47.705904 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:47.705909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:47.705928 (+ 19us) server_negotiation.cc:407] Connection header received
0806 20:24:47.706110 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:47.706114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:47.706175 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:47.706251 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:47.707686 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.708402 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.709787 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:47.710162 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:47.710252 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:47.714895 (+ 4643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:47.714911 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:47.714927 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:47.714972 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:47.718246 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.718968 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.718977 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.718979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.719117 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:47.719518 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:47.719522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:47.719525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:47.720061 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:47.720319 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:47.720618 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 20:24:47.720833 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:24:47.721894 18928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:47.811138 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 18796
I20260806 20:24:47.830650 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 18412
I20260806 20:24:47.831931 18481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:47.941039 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 18412
2026-08-06T20:24:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4430 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:24:48 dist-test-slave-6k22 krb5kdc[18986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:24:48 dist-test-slave-6k22 krb5kdc[18986](info): set up 2 sockets
Aug 06 20:24:48 dist-test-slave-6k22 krb5kdc[18986](info): commencing operation
krb5kdc: starting...
W20260806 20:24:49.197850 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260806 20:24:49.349361 17847 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.151s user 0.000s sys 0.007s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260806 20:24:49.673048 17847 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.323s user 0.000s sys 0.007s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260806 20:24:49.831270 17847 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.158s user 0.001s sys 0.006s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/test-user.keytab.
W20260806 20:24:50.397221 17847 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.566s user 0.004s sys 0.003s
Aug 06 20:24:50 dist-test-slave-6k22 krb5kdc[18986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:24:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:24:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
W20260806 20:24:50.570789 17847 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.17.109.254: real 0.114s user 0.001s sys 0.010s
I20260806 20:24:50.571907 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:40849
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:40849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:50.921020 18999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:50.921370 18999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:50.921566 18999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:50.929889 18999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:24:50.930020 18999 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 20:24:50.930141 18999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:50.930270 18999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:50.930380 18999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:24:50.930512 18999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:24:50.941192 18999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:40849
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:40849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:50.942513 18999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:50.943976 18999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:50.956539 19007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:50.956598 19004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:50.956605 19005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:50.957528 18999 server_base.cc:1034] running on GCE node
I20260806 20:24:50.958359 18999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:50.959841 18999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:50.961072 18999 hybrid_clock.cc:648] HybridClock initialized: now 1786047890961008 us; error 33 us; skew 500 ppm
I20260806 20:24:50.961428 18999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:50 dist-test-slave-6k22 krb5kdc[18986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047890, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:50.969708 18999 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:24:50.971478 18999 webserver.cc:460] Webserver started at http://127.17.109.254:33371/ using document root <none> and password file <none>
I20260806 20:24:50.972273 18999 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:50.972420 18999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:50.972771 18999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:50.975999 18999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "87848c5e728d429c90ac459e73ed02c9"
format_stamp: "Formatted at 2026-08-06 20:24:50 on dist-test-slave-6k22"
server_key: "b47831ae35abcdc89863d704c092406e"
server_key_iv: "b57a98caa108650860d3cafa6b83fcf4"
server_key_version: "encryptionkey@0"
I20260806 20:24:50.976842 18999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "87848c5e728d429c90ac459e73ed02c9"
format_stamp: "Formatted at 2026-08-06 20:24:50 on dist-test-slave-6k22"
server_key: "b47831ae35abcdc89863d704c092406e"
server_key_iv: "b57a98caa108650860d3cafa6b83fcf4"
server_key_version: "encryptionkey@0"
I20260806 20:24:50.982280 18999 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:24:50.986076 19014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:50.987857 18999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:24:50.988098 18999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "87848c5e728d429c90ac459e73ed02c9"
format_stamp: "Formatted at 2026-08-06 20:24:50 on dist-test-slave-6k22"
server_key: "b47831ae35abcdc89863d704c092406e"
server_key_iv: "b57a98caa108650860d3cafa6b83fcf4"
server_key_version: "encryptionkey@0"
I20260806 20:24:50.988399 18999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:51.017791 18999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:51.022858 18999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:51.023175 18999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:51.049563 18999 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:40849
I20260806 20:24:51.049600 19080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:40849 every 8 connection(s)
I20260806 20:24:51.051213 18999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:24:51.054294 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 18999
I20260806 20:24:51.054525 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:24:51.055246 17847 external_mini_cluster.cc:1428] Setting key 9e521b841f81e7e2b249fd2eeab86a44
I20260806 20:24:51.055716 19081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 20:24:51 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:51.072727 19081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9: Bootstrap starting.
I20260806 20:24:51.076709 19081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:51.078048 19081 log.cc:824] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:51.082032 19081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9: No bootstrap required, opened a new log
I20260806 20:24:51.082856 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:51.057400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.17.109.254:40849)
0806 20:24:51.058266 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:51.058279 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:51.058323 (+ 44us) server_negotiation.cc:407] Connection header received
0806 20:24:51.061084 (+ 2761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:51.061111 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:51.061568 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:51.061943 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:51.064085 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.065455 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.067456 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.068304 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.068454 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:51.074155 (+ 5701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:51.074163 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:51.074179 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:51.074229 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:51.077642 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:51.078715 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:51.078727 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:51.078740 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:51.079059 (+ 319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:51.079599 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:51.079605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:51.079609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:51.080353 (+ 744us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:51.080625 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:51.081168 (+ 543us) server_negotiation.cc:299] Negotiation successful
0806 20:24:51.081617 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":429,"threads_started":1}
I20260806 20:24:51.089602 19081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87848c5e728d429c90ac459e73ed02c9" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40849 } }
I20260806 20:24:51.090019 19081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:51.090167 19081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87848c5e728d429c90ac459e73ed02c9, State: Initialized, Role: FOLLOWER
I20260806 20:24:51.090785 19081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87848c5e728d429c90ac459e73ed02c9" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40849 } }
I20260806 20:24:51.091044 19081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:24:51.091184 19081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:24:51.091364 19081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:51.093755 19081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87848c5e728d429c90ac459e73ed02c9" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40849 } }
I20260806 20:24:51.094301 19081 leader_election.cc:302] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87848c5e728d429c90ac459e73ed02c9; no voters:
I20260806 20:24:51.094966 19081 leader_election.cc:288] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:24:51.095090 19086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:51.095999 19086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [term 1 LEADER]: Becoming Leader. State: Replica: 87848c5e728d429c90ac459e73ed02c9, State: Running, Role: LEADER
I20260806 20:24:51.096645 19086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87848c5e728d429c90ac459e73ed02c9" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40849 } }
I20260806 20:24:51.097472 19081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:24:51.099586 19087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87848c5e728d429c90ac459e73ed02c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87848c5e728d429c90ac459e73ed02c9" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40849 } } }
I20260806 20:24:51.099979 19087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [sys.catalog]: This master's current role is: LEADER
I20260806 20:24:51.103856 19088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87848c5e728d429c90ac459e73ed02c9. Latest consensus state: current_term: 1 leader_uuid: "87848c5e728d429c90ac459e73ed02c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87848c5e728d429c90ac459e73ed02c9" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40849 } } }
I20260806 20:24:51.104264 19088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9 [sys.catalog]: This master's current role is: LEADER
I20260806 20:24:51.104964 19094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:24:51.110453 19094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:24:51.119132 19094 catalog_manager.cc:1347] Generated new cluster ID: 578a6a513aac4cd48b06e941b5dcb27c
I20260806 20:24:51.119257 19094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:24:51.130882 19094 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:24:51.132735 19094 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:24:51.142736 19094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9: Generated new TSK 0
I20260806 20:24:51.143772 19094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:24:51.195506 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40849
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:51.533386 19108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:51.533666 19108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:51.533840 19108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:51.541364 19108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:51.541471 19108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:51.541605 19108 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:24:51.541702 19108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:24:51.552073 19108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:40849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:51.553381 19108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:51.554780 19108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:51.567998 19113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:51.568097 19114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:51.568116 19116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:51.569152 19108 server_base.cc:1034] running on GCE node
I20260806 20:24:51.569783 19108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:51.570638 19108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:51.571864 19108 hybrid_clock.cc:648] HybridClock initialized: now 1786047891571807 us; error 26 us; skew 500 ppm
I20260806 20:24:51.572127 19108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:51 dist-test-slave-6k22 krb5kdc[18986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047891, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:51.575492 19108 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:24:51.577206 19108 webserver.cc:460] Webserver started at http://127.17.109.193:43255/ using document root <none> and password file <none>
I20260806 20:24:51.577986 19108 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:51.578105 19108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:51.578490 19108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:51.581703 19108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "e5c68f3a90dd43df94035de3bff64b1f"
format_stamp: "Formatted at 2026-08-06 20:24:51 on dist-test-slave-6k22"
server_key: "b330fd404f918279b40daaefa418f071"
server_key_iv: "7eacb9679f28bde3ecc2016c6dbcee9a"
server_key_version: "encryptionkey@0"
I20260806 20:24:51.582561 19108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "e5c68f3a90dd43df94035de3bff64b1f"
format_stamp: "Formatted at 2026-08-06 20:24:51 on dist-test-slave-6k22"
server_key: "b330fd404f918279b40daaefa418f071"
server_key_iv: "7eacb9679f28bde3ecc2016c6dbcee9a"
server_key_version: "encryptionkey@0"
I20260806 20:24:51.588364 19108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 20:24:51.592299 19124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:51.593842 19108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 20:24:51.594040 19108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "e5c68f3a90dd43df94035de3bff64b1f"
format_stamp: "Formatted at 2026-08-06 20:24:51 on dist-test-slave-6k22"
server_key: "b330fd404f918279b40daaefa418f071"
server_key_iv: "7eacb9679f28bde3ecc2016c6dbcee9a"
server_key_version: "encryptionkey@0"
I20260806 20:24:51.594292 19108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:51.612372 19108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:51.616914 19108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:51.617229 19108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:51.620306 19108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:51.620417 19108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:51.620594 19108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:51.620662 19108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:24:51 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047891, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:51.644403 19132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:51.620755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:57347 (local address 127.17.109.254:40849)
0806 20:24:51.622657 (+ 1902us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:51.622667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:51.623351 (+ 684us) server_negotiation.cc:407] Connection header received
0806 20:24:51.625398 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:51.625404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:51.625491 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:51.625619 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:51.627496 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.628572 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.631203 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.631819 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.631965 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:51.636789 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:51.636794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:51.636797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:51.636829 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:51.639947 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:51.640921 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:51.640928 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:51.640934 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:51.641061 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:51.641719 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:51.641728 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:51.641730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:51.642021 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:51.642196 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:51.643769 (+ 1573us) server_negotiation.cc:299] Negotiation successful
0806 20:24:51.644006 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1747,"thread_start_us":1649,"threads_started":1}
I20260806 20:24:51.645006 19134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:51.621544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40849 (local address 127.17.109.193:57347)
0806 20:24:51.623196 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:51.623236 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:51.625119 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:51.625867 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:51.625875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:51.626454 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:51.627231 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:51.627259 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.628779 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.628785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:51.630917 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:51.630929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.632153 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.632166 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:51.632588 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:51.633477 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:51.633528 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:51.636550 (+ 3022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:51.640094 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:51.640102 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:51.640122 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:51.640791 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:51.641182 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:51.641191 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:51.641194 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:51.641537 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:51.642180 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:51.642188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:51.642647 (+ 459us) client_negotiation.cc:769] Sending connection context
0806 20:24:51.643037 (+ 390us) client_negotiation.cc:240] Negotiation successful
0806 20:24:51.643484 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1414,"thread_start_us":1157,"threads_started":1}
I20260806 20:24:51.668644 19132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:51.652044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:59075 (local address 127.17.109.254:40849)
0806 20:24:51.652314 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:51.652325 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:51.652918 (+ 593us) server_negotiation.cc:407] Connection header received
0806 20:24:51.653074 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:51.653084 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:51.653210 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:51.653362 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:51.654324 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.655902 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.657944 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.658598 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.658751 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:51.661912 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:51.661917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:51.661920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:51.661953 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:51.665336 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:51.666463 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:51.666467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:51.666469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:51.666602 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:51.667066 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:51.667072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:51.667075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:51.667347 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:51.667505 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:51.668026 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 20:24:51.668261 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 20:24:51.668794 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:51.652113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40849 (local address 127.17.109.193:59075)
0806 20:24:51.652813 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:51.652829 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:51.652948 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:51.653402 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:51.653406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:51.653859 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:51.654144 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:51.654153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.656064 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.656076 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:51.657777 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:51.657790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:51.658805 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:51.658812 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:51.659071 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:51.659983 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:51.660024 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:51.661717 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:51.665507 (+ 3790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:51.665517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:51.665525 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:51.666314 (+ 789us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:51.666796 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:51.666802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:51.666806 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:51.666949 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:51.667468 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:51.667473 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:51.667734 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 20:24:51.667918 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 20:24:51.668316 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":362,"threads_started":1}
I20260806 20:24:51.684777 19108 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:33207
I20260806 20:24:51.684799 19249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:33207 every 8 connection(s)
I20260806 20:24:51.686216 19108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:24:51.687752 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 19108
I20260806 20:24:51.687980 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:24:51.688871 17847 external_mini_cluster.cc:1428] Setting key 991ad76a65bba8539e2780c58e32da5b
I20260806 20:24:51.688892 19250 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40849
I20260806 20:24:51.689147 19250 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:51.689972 19250 heartbeater.cc:511] Master 127.17.109.254:40849 requested a full tablet report, sending...
I20260806 20:24:51.692274 19032 ts_manager.cc:194] Registered new tserver with Master: e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207)
I20260806 20:24:51.694171 19032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:57347
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:24:51.739209 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40849
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:52.077054 19255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:52.077373 19255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:52.077553 19255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:52.084986 19255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:52.085107 19255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:52.085263 19255 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:24:52.085358 19255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:24:52.095640 19255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:40849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:52.096948 19255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:52.098434 19255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:52.112211 19262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:52.112161 19263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:52.112249 19267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:52.113198 19255 server_base.cc:1034] running on GCE node
I20260806 20:24:52.113849 19255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:52.114746 19255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:52.115985 19255 hybrid_clock.cc:648] HybridClock initialized: now 1786047892115913 us; error 50 us; skew 500 ppm
I20260806 20:24:52.116282 19255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:52 dist-test-slave-6k22 krb5kdc[18986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:52.119530 19255 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:24:52.120957 19255 webserver.cc:460] Webserver started at http://127.17.109.194:39487/ using document root <none> and password file <none>
I20260806 20:24:52.121598 19255 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:52.121682 19255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:52.121950 19255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:52.124750 19255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "3abdccf167e34dd4b7a50a2e1d671386"
format_stamp: "Formatted at 2026-08-06 20:24:52 on dist-test-slave-6k22"
server_key: "2ba1f0fe9397bf52574d3411354d7eb9"
server_key_iv: "047c8454af7cb05238bca511edbb9d35"
server_key_version: "encryptionkey@0"
I20260806 20:24:52.125480 19255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "3abdccf167e34dd4b7a50a2e1d671386"
format_stamp: "Formatted at 2026-08-06 20:24:52 on dist-test-slave-6k22"
server_key: "2ba1f0fe9397bf52574d3411354d7eb9"
server_key_iv: "047c8454af7cb05238bca511edbb9d35"
server_key_version: "encryptionkey@0"
I20260806 20:24:52.130591 19255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:24:52.134127 19274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:52.135499 19255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 20:24:52.135752 19255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "3abdccf167e34dd4b7a50a2e1d671386"
format_stamp: "Formatted at 2026-08-06 20:24:52 on dist-test-slave-6k22"
server_key: "2ba1f0fe9397bf52574d3411354d7eb9"
server_key_iv: "047c8454af7cb05238bca511edbb9d35"
server_key_version: "encryptionkey@0"
I20260806 20:24:52.135934 19255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:52.170047 19255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:52.175482 19255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:52.175848 19255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:52.179551 19255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:52.179731 19255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 20:24:52.181679 19255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:52.181830 19255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
Aug 06 20:24:52 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:52.209673 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.180151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:39173 (local address 127.17.109.254:40849)
0806 20:24:52.181757 (+ 1606us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:52.181767 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:52.184510 (+ 2743us) server_negotiation.cc:407] Connection header received
0806 20:24:52.188032 (+ 3522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:52.188042 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:52.188156 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:52.188255 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:52.191366 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.192752 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.196117 (+ 3365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.196674 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.196832 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.202097 (+ 5265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:52.202103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:52.202107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:52.202168 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:52.205532 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.206799 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.206806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.206810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.206986 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.207469 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.207475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.207479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.207909 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:52.208086 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:52.209214 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0806 20:24:52.209392 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1475,"thread_start_us":1376,"threads_started":1}
I20260806 20:24:52.211864 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.180826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40849 (local address 127.17.109.194:39173)
0806 20:24:52.184286 (+ 3460us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:52.184344 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:52.187583 (+ 3239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:52.188923 (+ 1340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:52.188938 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:52.189978 (+ 1040us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:52.191080 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.191109 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.193008 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.193016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.195878 (+ 2862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.195902 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.196987 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.196998 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.197390 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.198511 (+ 1121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:52.198551 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:52.201844 (+ 3293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:52.205702 (+ 3858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:52.205718 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:52.205741 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:52.206602 (+ 861us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:52.207122 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:52.207125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:52.207128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:52.207321 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:52.208097 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:52.208108 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:52.208712 (+ 604us) client_negotiation.cc:769] Sending connection context
0806 20:24:52.209090 (+ 378us) client_negotiation.cc:240] Negotiation successful
0806 20:24:52.209809 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3130,"thread_start_us":2906,"threads_started":1}
I20260806 20:24:52.236842 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.218627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:46431 (local address 127.17.109.254:40849)
0806 20:24:52.218827 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:52.218832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:52.219575 (+ 743us) server_negotiation.cc:407] Connection header received
0806 20:24:52.219921 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:52.219927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:52.220007 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:52.220121 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:52.221712 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.222895 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.225405 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.226134 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.226319 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.229031 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:52.229041 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:52.229049 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:52.229099 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:52.233111 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.234217 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.234224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.234229 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.234396 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.234907 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.234913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.234917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.235484 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:52.235716 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:52.236198 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 20:24:52.236441 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 20:24:52.236949 19346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.218731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40849 (local address 127.17.109.194:46431)
0806 20:24:52.219428 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:52.219444 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:52.219748 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:52.220160 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:52.220165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:52.220967 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:52.221518 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.221540 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.223063 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.223069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.225166 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.225192 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.226412 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.226420 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.226602 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.227428 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:52.227446 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:52.228800 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:52.233303 (+ 4503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:52.233316 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:52.233321 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:52.234055 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:52.234564 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:52.234572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:52.234582 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:52.234765 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:52.235678 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:52.235683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:52.235900 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 20:24:52.236054 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 20:24:52.236423 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":400,"threads_started":1}
I20260806 20:24:52.246196 19255 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:37207
I20260806 20:24:52.246220 19401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:37207 every 8 connection(s)
I20260806 20:24:52.247431 19255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:24:52.249671 19403 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40849
I20260806 20:24:52.249866 19403 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:52.250432 19403 heartbeater.cc:511] Master 127.17.109.254:40849 requested a full tablet report, sending...
I20260806 20:24:52.251662 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 19255
I20260806 20:24:52.251879 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:24:52.251914 19032 ts_manager.cc:194] Registered new tserver with Master: 3abdccf167e34dd4b7a50a2e1d671386 (127.17.109.194:37207)
I20260806 20:24:52.252409 17847 external_mini_cluster.cc:1428] Setting key 018bdad4b9bd95787d671e3b1f675493
I20260806 20:24:52.252813 19032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:39173
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:24:52.300014 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40849
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:52.636148 19407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:52.636438 19407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:52.636601 19407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:52.644290 19407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:52.644404 19407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:52.644577 19407 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:24:52.644701 19407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:24:52.654759 19407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:40849
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:52.656112 19407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:52.657493 19407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:52.671432 19412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:52.671430 19414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:52.671404 19417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:52.672395 19407 server_base.cc:1034] running on GCE node
I20260806 20:24:52.673038 19407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:52.673951 19407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:52.675249 19407 hybrid_clock.cc:648] HybridClock initialized: now 1786047892675172 us; error 50 us; skew 500 ppm
I20260806 20:24:52.675726 19407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:52 dist-test-slave-6k22 krb5kdc[18986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:52.679184 19407 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:24:52.680783 19407 webserver.cc:460] Webserver started at http://127.17.109.195:42435/ using document root <none> and password file <none>
I20260806 20:24:52.681442 19407 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:52.681531 19407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:52.681823 19407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:52.684844 19407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "c32b96fe938f419ba4b680ad6350c8c5"
format_stamp: "Formatted at 2026-08-06 20:24:52 on dist-test-slave-6k22"
server_key: "12a3c07ff571485ed5ef0b94d9a2cfe3"
server_key_iv: "1fdc7627a9b74dc3183b954448e722e6"
server_key_version: "encryptionkey@0"
I20260806 20:24:52.685695 19407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "c32b96fe938f419ba4b680ad6350c8c5"
format_stamp: "Formatted at 2026-08-06 20:24:52 on dist-test-slave-6k22"
server_key: "12a3c07ff571485ed5ef0b94d9a2cfe3"
server_key_iv: "1fdc7627a9b74dc3183b954448e722e6"
server_key_version: "encryptionkey@0"
I20260806 20:24:52.690903 19407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 20:24:52.694731 19424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:52.696180 19407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:24:52.696384 19407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "c32b96fe938f419ba4b680ad6350c8c5"
format_stamp: "Formatted at 2026-08-06 20:24:52 on dist-test-slave-6k22"
server_key: "12a3c07ff571485ed5ef0b94d9a2cfe3"
server_key_iv: "1fdc7627a9b74dc3183b954448e722e6"
server_key_version: "encryptionkey@0"
I20260806 20:24:52.696630 19407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:52.697356 19250 heartbeater.cc:503] Master 127.17.109.254:40849 was elected leader, sending a full tablet report...
I20260806 20:24:52.712493 19407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:52.717610 19407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:52.717931 19407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:52.720801 19407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:52.720952 19407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:52.721100 19407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:52.721181 19407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:24:52 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:52.749765 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.721682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:56827 (local address 127.17.109.254:40849)
0806 20:24:52.721979 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:52.721988 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:52.723355 (+ 1367us) server_negotiation.cc:407] Connection header received
0806 20:24:52.725752 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:52.725758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:52.725848 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:52.725964 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:52.728146 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.729282 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.732960 (+ 3678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.733481 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.733628 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.740060 (+ 6432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:52.740070 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:52.740078 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:52.740133 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:52.744910 (+ 4777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.746295 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.746301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.746306 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.746457 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.747213 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.747221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.747226 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.747721 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:52.747937 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:52.749212 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0806 20:24:52.749416 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260806 20:24:52.752048 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.722237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40849 (local address 127.17.109.195:56827)
0806 20:24:52.723169 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:52.723218 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:52.725433 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:52.726226 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:52.726240 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:52.727011 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:52.727938 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.727965 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.729486 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.729493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.732740 (+ 3247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.732762 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.733867 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.733885 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.734279 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.735468 (+ 1189us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:52.735509 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:52.739667 (+ 4158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:52.745158 (+ 5491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:52.745173 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:52.745202 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:52.746139 (+ 937us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:52.746593 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:52.746601 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:52.746606 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:52.747014 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:52.747908 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:52.747916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:52.748599 (+ 683us) client_negotiation.cc:769] Sending connection context
0806 20:24:52.749037 (+ 438us) client_negotiation.cc:240] Negotiation successful
0806 20:24:52.749854 (+ 817us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":386,"threads_started":1}
I20260806 20:24:52.773185 19407 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:33175
I20260806 20:24:52.773280 19546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:33175 every 8 connection(s)
I20260806 20:24:52.774745 19407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:24:52.777194 19547 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40849
I20260806 20:24:52.777449 19547 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:52.777736 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.760238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:37785 (local address 127.17.109.254:40849)
0806 20:24:52.760442 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:52.760450 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:52.761474 (+ 1024us) server_negotiation.cc:407] Connection header received
0806 20:24:52.761790 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:52.761796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:52.761906 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:52.762031 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:52.763664 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.764646 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.766778 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.767685 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.767883 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.770914 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:52.770918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:52.770921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:52.770953 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:52.774179 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.775083 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.775087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.775089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.775281 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.775940 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.775945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.775948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.776376 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:52.776553 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:52.777089 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 20:24:52.777384 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:24:52.777843 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.760470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40849 (local address 127.17.109.195:37785)
0806 20:24:52.761342 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:52.761368 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:52.761595 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:52.762055 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:52.762059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:52.763004 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:52.763462 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.763477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.764844 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.764848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.766575 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.766598 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.767963 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.767972 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.768197 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.769042 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:52.769082 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:52.770714 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:52.774325 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:52.774331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:52.774334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:52.774937 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:52.775438 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:52.775446 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:52.775455 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:52.775719 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:52.776496 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:52.776500 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:52.776798 (+ 298us) client_negotiation.cc:769] Sending connection context
0806 20:24:52.776938 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 20:24:52.777307 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":550,"threads_started":1}
I20260806 20:24:52.778553 19547 heartbeater.cc:511] Master 127.17.109.254:40849 requested a full tablet report, sending...
I20260806 20:24:52.779984 19032 ts_manager.cc:194] Registered new tserver with Master: c32b96fe938f419ba4b680ad6350c8c5 (127.17.109.195:33175)
I20260806 20:24:52.780941 19032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:56827
I20260806 20:24:52.782912 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 19407
I20260806 20:24:52.783087 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:24:52.783638 17847 external_mini_cluster.cc:1428] Setting key 3889ea55df5b6274ffc521bef388e5c9
I20260806 20:24:52.786501 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 20:24:52 dist-test-slave-6k22 krb5kdc[18986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 20:24:52 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:52.837625 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.823427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58774 (local address 127.17.109.254:40849)
0806 20:24:52.823656 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:52.823661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:52.823917 (+ 256us) server_negotiation.cc:407] Connection header received
0806 20:24:52.824075 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:52.824079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:52.824146 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:52.824237 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:52.825686 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.826430 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.827815 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.828282 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.828419 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.832358 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:52.832363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:52.832374 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:52.832411 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:52.835134 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.835807 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.835811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.835813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.835943 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:52.836338 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:52.836342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:52.836344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:52.836644 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:52.836852 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:52.837188 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 20:24:52.837371 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 20:24:52.842309 19032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:24:52.850263 19032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:24:52.883988 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.863161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36184 (local address 127.17.109.193:33207)
0806 20:24:52.876202 (+ 13041us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:52.876213 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:52.876246 (+ 33us) server_negotiation.cc:407] Connection header received
0806 20:24:52.876391 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:52.876398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:52.876645 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:52.877736 (+ 1091us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:52.877953 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.879177 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.881632 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.882595 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.882801 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.882985 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:52.883446 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 20:24:52.883652 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12843,"thread_start_us":12681,"threads_started":1}
I20260806 20:24:52.884001 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.862806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:33207 (local address 127.0.0.1:36184)
0806 20:24:52.869388 (+ 6582us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:52.869441 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:52.869668 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:52.876914 (+ 7246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:52.876919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:52.876974 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:52.877565 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.877578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.879308 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.879317 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.881468 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.881480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.882830 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.882837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.883087 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.883146 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 20:24:52.883293 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 20:24:52.883465 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6246,"mutex_wait_us":28,"spinlock_wait_cycles":3328,"thread_start_us":672,"threads_started":1}
I20260806 20:24:52.886034 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.865804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.17.109.195:33175)
0806 20:24:52.877333 (+ 11529us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:52.877341 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:52.877377 (+ 36us) server_negotiation.cc:407] Connection header received
0806 20:24:52.877484 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:52.877493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:52.877717 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:52.877949 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:52.878584 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.880373 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.883359 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.884851 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.885076 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.885257 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:52.885494 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 20:24:52.885664 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11335,"thread_start_us":11487,"threads_started":1}
I20260806 20:24:52.886050 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.866471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33175 (local address 127.0.0.1:55508)
0806 20:24:52.876576 (+ 10105us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:52.876596 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:52.876715 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:52.877938 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:52.877942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:52.877966 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:52.878349 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.878360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.880575 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.880583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.883193 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.883204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.885074 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.885081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.885284 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.885326 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 20:24:52.885439 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 20:24:52.885573 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9939,"mutex_wait_us":60,"thread_start_us":9773,"threads_started":1}
I20260806 20:24:52.887064 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.863396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:37207 (local address 127.0.0.1:55144)
0806 20:24:52.875433 (+ 12037us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:52.875456 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:52.875604 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:52.876893 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:52.876903 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:52.876963 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:52.877949 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.877964 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.880167 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.880173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.882964 (+ 2791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:52.882983 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.886273 (+ 3290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.886280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:52.886460 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.886491 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 20:24:52.886589 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 20:24:52.886721 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11885,"mutex_wait_us":300,"spinlock_wait_cycles":4480,"thread_start_us":11693,"threads_started":1}
I20260806 20:24:52.887804 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:52.863911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.17.109.194:37207)
0806 20:24:52.875836 (+ 11925us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:52.875849 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:52.875884 (+ 35us) server_negotiation.cc:407] Connection header received
0806 20:24:52.875993 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:52.876002 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:52.876331 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:52.876609 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:52.878294 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.879981 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.883146 (+ 3165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:52.885761 (+ 2615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:52.885937 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:52.886104 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:52.887304 (+ 1200us) server_negotiation.cc:299] Negotiation successful
0806 20:24:52.887467 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11695,"thread_start_us":11551,"threads_started":1}
I20260806 20:24:52.888622 19179 tablet_service.cc:1467] Processing CreateTablet for tablet d9b080f6f9ff436abced7d321a622835 (DEFAULT_TABLE table=test-table [id=32bc5d75aa31451484a797ddb7bee35c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:52.890218 19179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b080f6f9ff436abced7d321a622835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:52.890033 19475 tablet_service.cc:1467] Processing CreateTablet for tablet d9b080f6f9ff436abced7d321a622835 (DEFAULT_TABLE table=test-table [id=32bc5d75aa31451484a797ddb7bee35c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:52.891386 19475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b080f6f9ff436abced7d321a622835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:52.891266 19329 tablet_service.cc:1467] Processing CreateTablet for tablet d9b080f6f9ff436abced7d321a622835 (DEFAULT_TABLE table=test-table [id=32bc5d75aa31451484a797ddb7bee35c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:52.892609 19329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b080f6f9ff436abced7d321a622835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:52.904624 19568 tablet_bootstrap.cc:492] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Bootstrap starting.
I20260806 20:24:52.907178 19569 tablet_bootstrap.cc:492] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Bootstrap starting.
I20260806 20:24:52.908253 19570 tablet_bootstrap.cc:492] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Bootstrap starting.
I20260806 20:24:52.909329 19568 tablet_bootstrap.cc:654] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:52.910737 19568 log.cc:824] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:52.911365 19570 tablet_bootstrap.cc:654] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:52.911438 19569 tablet_bootstrap.cc:654] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:52.912633 19569 log.cc:824] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:52.912698 19570 log.cc:824] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:52.914369 19568 tablet_bootstrap.cc:492] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: No bootstrap required, opened a new log
I20260806 20:24:52.914637 19568 ts_tablet_manager.cc:1397] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 20:24:52.915951 19570 tablet_bootstrap.cc:492] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: No bootstrap required, opened a new log
I20260806 20:24:52.916322 19570 ts_tablet_manager.cc:1397] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260806 20:24:52.916889 19569 tablet_bootstrap.cc:492] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: No bootstrap required, opened a new log
I20260806 20:24:52.917179 19569 ts_tablet_manager.cc:1397] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260806 20:24:52.923481 19568 raft_consensus.cc:348] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:52.923951 19568 raft_consensus.cc:374] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:52.924074 19568 raft_consensus.cc:729] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32b96fe938f419ba4b680ad6350c8c5, State: Initialized, Role: FOLLOWER
I20260806 20:24:52.924706 19568 consensus_queue.cc:260] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:52.925915 19547 heartbeater.cc:503] Master 127.17.109.254:40849 was elected leader, sending a full tablet report...
I20260806 20:24:52.926664 19568 ts_tablet_manager.cc:1428] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260806 20:24:52.927929 19570 raft_consensus.cc:348] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:52.928468 19570 raft_consensus.cc:374] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:52.928244 19569 raft_consensus.cc:348] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:52.928663 19570 raft_consensus.cc:729] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abdccf167e34dd4b7a50a2e1d671386, State: Initialized, Role: FOLLOWER
I20260806 20:24:52.928720 19569 raft_consensus.cc:374] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:52.928833 19569 raft_consensus.cc:729] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c68f3a90dd43df94035de3bff64b1f, State: Initialized, Role: FOLLOWER
I20260806 20:24:52.929427 19569 consensus_queue.cc:260] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:52.929426 19570 consensus_queue.cc:260] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:52.930651 19403 heartbeater.cc:503] Master 127.17.109.254:40849 was elected leader, sending a full tablet report...
I20260806 20:24:52.931458 19570 ts_tablet_manager.cc:1428] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260806 20:24:52.931967 19569 ts_tablet_manager.cc:1428] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260806 20:24:52.938764 19252 tablet.cc:2369] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:24:52.999727 19404 tablet.cc:2369] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:24:53.026695 19548 tablet.cc:2369] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:24:53.097342 19574 raft_consensus.cc:482] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:24:53.097751 19574 raft_consensus.cc:504] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.099884 19574 leader_election.cc:288] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207), 3abdccf167e34dd4b7a50a2e1d671386 (127.17.109.194:37207)
I20260806 20:24:53.107354 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.100468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:50385 (local address 127.17.109.193:33207)
0806 20:24:53.100768 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.100777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.100827 (+ 50us) server_negotiation.cc:407] Connection header received
0806 20:24:53.101101 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.101109 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.101216 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.101323 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:53.101833 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.102905 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.105341 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.106525 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.106693 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.106773 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.107019 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.107143 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260806 20:24:53.107424 19433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.100436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:33207 (local address 127.17.109.195:50385)
0806 20:24:53.100731 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:53.100754 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:53.100915 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:53.101372 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:53.101377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:53.101409 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:53.101663 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.101680 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.103029 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.103033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.105183 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.105198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.106685 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.106689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.106820 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.106845 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 20:24:53.106964 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 20:24:53.107104 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":32}
I20260806 20:24:53.107760 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.100499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:37839 (local address 127.17.109.194:37207)
0806 20:24:53.100837 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.100844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.101473 (+ 629us) server_negotiation.cc:407] Connection header received
0806 20:24:53.101664 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.101668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.101724 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.101827 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:53.102308 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.103263 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.105603 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.106803 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.106956 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.107029 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.107364 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.107513 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260806 20:24:53.107932 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.100590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:37207 (local address 127.17.109.195:37839)
0806 20:24:53.101380 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:53.101398 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:53.101516 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:53.101855 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:53.101859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:53.101887 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:53.102149 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.102155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.103403 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.103408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.105459 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.105469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.107016 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.107022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.107173 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.107205 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 20:24:53.107302 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 20:24:53.107414 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":34,"thread_start_us":430,"threads_started":1}
I20260806 20:24:53.108767 19204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9b080f6f9ff436abced7d321a622835" candidate_uuid: "c32b96fe938f419ba4b680ad6350c8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c68f3a90dd43df94035de3bff64b1f" is_pre_election: true
I20260806 20:24:53.108987 19355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9b080f6f9ff436abced7d321a622835" candidate_uuid: "c32b96fe938f419ba4b680ad6350c8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abdccf167e34dd4b7a50a2e1d671386" is_pre_election: true
I20260806 20:24:53.109397 19204 raft_consensus.cc:2454] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32b96fe938f419ba4b680ad6350c8c5 in term 0.
I20260806 20:24:53.109613 19355 raft_consensus.cc:2454] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32b96fe938f419ba4b680ad6350c8c5 in term 0.
I20260806 20:24:53.110106 19425 leader_election.cc:302] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32b96fe938f419ba4b680ad6350c8c5, e5c68f3a90dd43df94035de3bff64b1f; no voters:
I20260806 20:24:53.110493 19574 raft_consensus.cc:2781] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:24:53.110656 19574 raft_consensus.cc:482] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:24:53.110791 19574 raft_consensus.cc:3037] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.113091 19574 raft_consensus.cc:504] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.113749 19574 leader_election.cc:288] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [CANDIDATE]: Term 1 election: Requested vote from peers e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207), 3abdccf167e34dd4b7a50a2e1d671386 (127.17.109.194:37207)
I20260806 20:24:53.114156 19204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9b080f6f9ff436abced7d321a622835" candidate_uuid: "c32b96fe938f419ba4b680ad6350c8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c68f3a90dd43df94035de3bff64b1f"
I20260806 20:24:53.114271 19355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9b080f6f9ff436abced7d321a622835" candidate_uuid: "c32b96fe938f419ba4b680ad6350c8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abdccf167e34dd4b7a50a2e1d671386"
I20260806 20:24:53.114400 19204 raft_consensus.cc:3037] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.114506 19355 raft_consensus.cc:3037] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.117009 19204 raft_consensus.cc:2454] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32b96fe938f419ba4b680ad6350c8c5 in term 1.
I20260806 20:24:53.117105 19355 raft_consensus.cc:2454] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32b96fe938f419ba4b680ad6350c8c5 in term 1.
I20260806 20:24:53.117468 19425 leader_election.cc:302] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32b96fe938f419ba4b680ad6350c8c5, e5c68f3a90dd43df94035de3bff64b1f; no voters:
I20260806 20:24:53.117877 19574 raft_consensus.cc:2781] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:53.118286 19574 raft_consensus.cc:686] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 LEADER]: Becoming Leader. State: Replica: c32b96fe938f419ba4b680ad6350c8c5, State: Running, Role: LEADER
I20260806 20:24:53.118758 19574 consensus_queue.cc:237] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.123533 19031 catalog_manager.cc:5495] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c32b96fe938f419ba4b680ad6350c8c5 (127.17.109.195). New cstate: current_term: 1 leader_uuid: "c32b96fe938f419ba4b680ad6350c8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } health_report { overall_health: HEALTHY } } }
I20260806 20:24:53.197247 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.191351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55522 (local address 127.17.109.195:33175)
0806 20:24:53.191622 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.191633 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.191652 (+ 19us) server_negotiation.cc:407] Connection header received
0806 20:24:53.191967 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.191971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.192047 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.192172 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:53.192580 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.193377 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.194993 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.195385 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.195478 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.196073 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:53.196632 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:53.196741 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.196857 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.196924 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 20:24:53.209448 19355 raft_consensus.cc:1261] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Refusing update from remote peer c32b96fe938f419ba4b680ad6350c8c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:53.210601 19578 consensus_queue.cc:1035] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:53.211400 19204 raft_consensus.cc:1261] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Refusing update from remote peer c32b96fe938f419ba4b680ad6350c8c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:53.212217 19578 consensus_queue.cc:1035] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:53.246043 19030 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:24:53.246511 19030 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:24:53.251968 19030 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9: Sending DeleteTablet for 3 replicas of tablet d9b080f6f9ff436abced7d321a622835
I20260806 20:24:53.253410 19179 tablet_service.cc:1514] Processing DeleteTablet for tablet d9b080f6f9ff436abced7d321a622835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:36184
I20260806 20:24:53.254648 19329 tablet_service.cc:1514] Processing DeleteTablet for tablet d9b080f6f9ff436abced7d321a622835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:55144
I20260806 20:24:53.254776 19592 tablet_replica.cc:330] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: stopping tablet replica
I20260806 20:24:53.255414 19592 raft_consensus.cc:2229] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:24:53.256013 19592 raft_consensus.cc:2258] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:53.256033 19475 tablet_service.cc:1514] Processing DeleteTablet for tablet d9b080f6f9ff436abced7d321a622835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:55508
I20260806 20:24:53.258345 19592 ts_tablet_manager.cc:1905] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:53.258625 19030 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:24:53.259601 19030 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:24:53.266882 19592 ts_tablet_manager.cc:1918] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:24:53.267207 19592 log.cc:1196] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/d9b080f6f9ff436abced7d321a622835
I20260806 20:24:53.267961 19592 ts_tablet_manager.cc:1939] T d9b080f6f9ff436abced7d321a622835 P e5c68f3a90dd43df94035de3bff64b1f: Deleting consensus metadata
I20260806 20:24:53.268605 19328 tablet_service.cc:1467] Processing CreateTablet for tablet 6797ba3d208d486aa4c55a51c2504ce1 (DEFAULT_TABLE table=test-table [id=2712f1f7326f463e8bc505221e4df110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:53.269238 19328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6797ba3d208d486aa4c55a51c2504ce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:53.272089 19015 catalog_manager.cc:4841] TS e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207): tablet d9b080f6f9ff436abced7d321a622835 (table test-table [id=32bc5d75aa31451484a797ddb7bee35c]) successfully deleted
I20260806 20:24:53.272404 19179 tablet_service.cc:1467] Processing CreateTablet for tablet 6797ba3d208d486aa4c55a51c2504ce1 (DEFAULT_TABLE table=test-table [id=2712f1f7326f463e8bc505221e4df110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:53.272857 19179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6797ba3d208d486aa4c55a51c2504ce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:53.275036 19570 tablet_bootstrap.cc:492] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: Bootstrap starting.
I20260806 20:24:53.276523 19474 tablet_service.cc:1467] Processing CreateTablet for tablet 6797ba3d208d486aa4c55a51c2504ce1 (DEFAULT_TABLE table=test-table [id=2712f1f7326f463e8bc505221e4df110]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:53.276751 19570 tablet_bootstrap.cc:654] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:53.277258 19474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6797ba3d208d486aa4c55a51c2504ce1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:53.279955 19569 tablet_bootstrap.cc:492] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: Bootstrap starting.
I20260806 20:24:53.281597 19569 tablet_bootstrap.cc:654] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:53.287209 19568 tablet_bootstrap.cc:492] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: Bootstrap starting.
I20260806 20:24:53.289661 19568 tablet_bootstrap.cc:654] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:53.292737 19594 tablet_replica.cc:330] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: stopping tablet replica
I20260806 20:24:53.293958 19593 tablet_replica.cc:330] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: stopping tablet replica
I20260806 20:24:53.293983 19569 tablet_bootstrap.cc:492] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: No bootstrap required, opened a new log
I20260806 20:24:53.294175 19569 ts_tablet_manager.cc:1397] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.010s
I20260806 20:24:53.294587 19593 raft_consensus.cc:2229] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:24:53.295224 19593 raft_consensus.cc:2258] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:53.295817 19594 raft_consensus.cc:2229] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:24:53.295482 19569 raft_consensus.cc:348] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.296226 19569 raft_consensus.cc:374] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:53.296394 19569 raft_consensus.cc:729] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c68f3a90dd43df94035de3bff64b1f, State: Initialized, Role: FOLLOWER
I20260806 20:24:53.296597 19594 raft_consensus.cc:2258] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:53.297619 19569 consensus_queue.cc:260] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.298712 19569 ts_tablet_manager.cc:1428] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260806 20:24:53.299687 19570 tablet_bootstrap.cc:492] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: No bootstrap required, opened a new log
I20260806 20:24:53.299857 19570 ts_tablet_manager.cc:1397] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.016s
I20260806 20:24:53.300402 19594 ts_tablet_manager.cc:1905] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:53.300503 19593 ts_tablet_manager.cc:1905] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:53.300693 19570 raft_consensus.cc:348] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.300916 19570 raft_consensus.cc:374] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:53.301090 19570 raft_consensus.cc:729] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abdccf167e34dd4b7a50a2e1d671386, State: Initialized, Role: FOLLOWER
I20260806 20:24:53.301518 19570 consensus_queue.cc:260] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.302270 19570 ts_tablet_manager.cc:1428] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 20:24:53.310176 19593 ts_tablet_manager.cc:1918] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:24:53.310477 19593 log.cc:1196] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/d9b080f6f9ff436abced7d321a622835
I20260806 20:24:53.311262 19593 ts_tablet_manager.cc:1939] T d9b080f6f9ff436abced7d321a622835 P 3abdccf167e34dd4b7a50a2e1d671386: Deleting consensus metadata
I20260806 20:24:53.312829 19568 tablet_bootstrap.cc:492] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: No bootstrap required, opened a new log
I20260806 20:24:53.313028 19568 ts_tablet_manager.cc:1397] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: Time spent bootstrapping tablet: real 0.026s user 0.008s sys 0.001s
I20260806 20:24:53.313606 19018 catalog_manager.cc:4841] TS 3abdccf167e34dd4b7a50a2e1d671386 (127.17.109.194:37207): tablet d9b080f6f9ff436abced7d321a622835 (table test-table [id=32bc5d75aa31451484a797ddb7bee35c]) successfully deleted
I20260806 20:24:53.314087 19568 raft_consensus.cc:348] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.314592 19568 raft_consensus.cc:374] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:53.314733 19568 raft_consensus.cc:729] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32b96fe938f419ba4b680ad6350c8c5, State: Initialized, Role: FOLLOWER
I20260806 20:24:53.315080 19568 consensus_queue.cc:260] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.315948 19568 ts_tablet_manager.cc:1428] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 20:24:53.316834 19594 ts_tablet_manager.cc:1918] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:24:53.317128 19594 log.cc:1196] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/d9b080f6f9ff436abced7d321a622835
I20260806 20:24:53.317888 19594 ts_tablet_manager.cc:1939] T d9b080f6f9ff436abced7d321a622835 P c32b96fe938f419ba4b680ad6350c8c5: Deleting consensus metadata
I20260806 20:24:53.320047 19015 catalog_manager.cc:4841] TS c32b96fe938f419ba4b680ad6350c8c5 (127.17.109.195:33175): tablet d9b080f6f9ff436abced7d321a622835 (table test-table [id=32bc5d75aa31451484a797ddb7bee35c]) successfully deleted
I20260806 20:24:53.342365 19574 raft_consensus.cc:482] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:24:53.342617 19574 raft_consensus.cc:504] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.343402 19574 leader_election.cc:288] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207), 3abdccf167e34dd4b7a50a2e1d671386 (127.17.109.194:37207)
I20260806 20:24:53.344009 19204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6797ba3d208d486aa4c55a51c2504ce1" candidate_uuid: "c32b96fe938f419ba4b680ad6350c8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c68f3a90dd43df94035de3bff64b1f" is_pre_election: true
I20260806 20:24:53.344054 19355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6797ba3d208d486aa4c55a51c2504ce1" candidate_uuid: "c32b96fe938f419ba4b680ad6350c8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abdccf167e34dd4b7a50a2e1d671386" is_pre_election: true
I20260806 20:24:53.344344 19355 raft_consensus.cc:2454] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32b96fe938f419ba4b680ad6350c8c5 in term 0.
I20260806 20:24:53.344332 19204 raft_consensus.cc:2454] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c32b96fe938f419ba4b680ad6350c8c5 in term 0.
I20260806 20:24:53.344842 19425 leader_election.cc:302] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c32b96fe938f419ba4b680ad6350c8c5, e5c68f3a90dd43df94035de3bff64b1f; no voters:
I20260806 20:24:53.345168 19574 raft_consensus.cc:2781] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:24:53.345293 19574 raft_consensus.cc:482] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:24:53.345407 19574 raft_consensus.cc:3037] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.348670 19574 raft_consensus.cc:504] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.349411 19574 leader_election.cc:288] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [CANDIDATE]: Term 1 election: Requested vote from peers e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207), 3abdccf167e34dd4b7a50a2e1d671386 (127.17.109.194:37207)
I20260806 20:24:53.349921 19204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6797ba3d208d486aa4c55a51c2504ce1" candidate_uuid: "c32b96fe938f419ba4b680ad6350c8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c68f3a90dd43df94035de3bff64b1f"
I20260806 20:24:53.350132 19204 raft_consensus.cc:3037] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.350257 19355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6797ba3d208d486aa4c55a51c2504ce1" candidate_uuid: "c32b96fe938f419ba4b680ad6350c8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3abdccf167e34dd4b7a50a2e1d671386"
I20260806 20:24:53.350492 19355 raft_consensus.cc:3037] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.352761 19355 raft_consensus.cc:2454] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32b96fe938f419ba4b680ad6350c8c5 in term 1.
I20260806 20:24:53.353190 19204 raft_consensus.cc:2454] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c32b96fe938f419ba4b680ad6350c8c5 in term 1.
I20260806 20:24:53.353257 19428 leader_election.cc:302] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abdccf167e34dd4b7a50a2e1d671386, c32b96fe938f419ba4b680ad6350c8c5; no voters:
I20260806 20:24:53.353611 19574 raft_consensus.cc:2781] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:53.353798 19574 raft_consensus.cc:686] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 LEADER]: Becoming Leader. State: Replica: c32b96fe938f419ba4b680ad6350c8c5, State: Running, Role: LEADER
I20260806 20:24:53.355259 19574 consensus_queue.cc:237] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.358107 19031 catalog_manager.cc:5495] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c32b96fe938f419ba4b680ad6350c8c5 (127.17.109.195). New cstate: current_term: 1 leader_uuid: "c32b96fe938f419ba4b680ad6350c8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } health_report { overall_health: HEALTHY } } }
Aug 06 20:24:53 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047890, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:53.412179 19604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.394087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40849 (local address 127.0.0.1:58780)
0806 20:24:53.394705 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:53.394732 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:53.394861 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:53.395532 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:53.395538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:53.396465 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:53.396744 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.396759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.398354 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.398359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.400006 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.400023 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.400784 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.400790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.400973 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.401770 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:53.401791 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:53.405078 (+ 3287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:53.409248 (+ 4170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:53.409261 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:53.409268 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:53.409915 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:53.410340 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:53.410344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:53.410346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:53.410512 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:53.411162 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:53.411169 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:53.411342 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 20:24:53.411489 (+ 147us) client_negotiation.cc:240] Negotiation successful
0806 20:24:53.411806 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":366,"threads_started":1}
I20260806 20:24:53.412539 19605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.394158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.17.109.254:40849)
0806 20:24:53.395246 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.395249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.395263 (+ 14us) server_negotiation.cc:407] Connection header received
0806 20:24:53.395302 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.395305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.395355 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.395440 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:53.396946 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.398201 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.400200 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.400609 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.400713 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.405299 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:53.405307 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:53.405312 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:53.405350 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:53.409085 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:53.410055 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:53.410061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:53.410065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:53.410221 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:53.410632 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:53.410637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:53.410640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:53.411014 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:53.411160 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.411649 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.412098 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":973,"thread_start_us":880,"threads_started":1}
I20260806 20:24:53.418459 19031 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:58780:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 20:24:53.419873 19031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:24:53.427356 19179 tablet_service.cc:1467] Processing CreateTablet for tablet c722261dedb744fe9a99f2b9c1bda456 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a7dd47e5549448fb9899e2fce82ef35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:24:53.428011 19179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c722261dedb744fe9a99f2b9c1bda456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:53.428112 19329 tablet_service.cc:1467] Processing CreateTablet for tablet c722261dedb744fe9a99f2b9c1bda456 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a7dd47e5549448fb9899e2fce82ef35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:24:53.428681 19329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c722261dedb744fe9a99f2b9c1bda456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:53.432523 19474 tablet_service.cc:1467] Processing CreateTablet for tablet c722261dedb744fe9a99f2b9c1bda456 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a7dd47e5549448fb9899e2fce82ef35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:24:53.433126 19474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c722261dedb744fe9a99f2b9c1bda456. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:53.434005 19569 tablet_bootstrap.cc:492] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: Bootstrap starting.
I20260806 20:24:53.435828 19569 tablet_bootstrap.cc:654] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:53.440969 19570 tablet_bootstrap.cc:492] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: Bootstrap starting.
I20260806 20:24:53.441071 19568 tablet_bootstrap.cc:492] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: Bootstrap starting.
I20260806 20:24:53.443151 19568 tablet_bootstrap.cc:654] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:53.443728 19570 tablet_bootstrap.cc:654] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:53.448318 19569 tablet_bootstrap.cc:492] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: No bootstrap required, opened a new log
I20260806 20:24:53.448522 19569 ts_tablet_manager.cc:1397] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260806 20:24:53.449546 19569 raft_consensus.cc:348] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.449820 19569 raft_consensus.cc:374] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:53.449914 19569 raft_consensus.cc:729] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c68f3a90dd43df94035de3bff64b1f, State: Initialized, Role: FOLLOWER
I20260806 20:24:53.450199 19569 consensus_queue.cc:260] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.450852 19576 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.451295 19576 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:53.452895 19569 ts_tablet_manager.cc:1428] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 20:24:53.456554 19570 tablet_bootstrap.cc:492] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: No bootstrap required, opened a new log
I20260806 20:24:53.456735 19570 ts_tablet_manager.cc:1397] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.007s
I20260806 20:24:53.457384 19570 raft_consensus.cc:348] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.457597 19570 raft_consensus.cc:374] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:53.457688 19570 raft_consensus.cc:729] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3abdccf167e34dd4b7a50a2e1d671386, State: Initialized, Role: FOLLOWER
I20260806 20:24:53.457958 19570 consensus_queue.cc:260] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.458577 19570 ts_tablet_manager.cc:1428] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 20:24:53.458477 19575 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.458897 19575 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:53.459467 19568 tablet_bootstrap.cc:492] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: No bootstrap required, opened a new log
I20260806 20:24:53.459694 19568 ts_tablet_manager.cc:1397] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.010s
I20260806 20:24:53.460598 19568 raft_consensus.cc:348] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.460877 19568 raft_consensus.cc:374] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:53.460978 19568 raft_consensus.cc:729] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32b96fe938f419ba4b680ad6350c8c5, State: Initialized, Role: FOLLOWER
I20260806 20:24:53.461252 19568 consensus_queue.cc:260] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.461750 19574 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.462051 19574 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:53.462543 19568 ts_tablet_manager.cc:1428] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 20:24:53.504011 19575 raft_consensus.cc:482] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:24:53.504244 19575 raft_consensus.cc:504] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.506136 19575 leader_election.cc:288] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207), c32b96fe938f419ba4b680ad6350c8c5 (127.17.109.195:33175)
I20260806 20:24:53.514922 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.506457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:42771 (local address 127.17.109.193:33207)
0806 20:24:53.506669 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.506677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.507163 (+ 486us) server_negotiation.cc:407] Connection header received
0806 20:24:53.507436 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.507445 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.507576 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.507673 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:53.508582 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.509710 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.511873 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.513340 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.513520 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.513596 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.514290 (+ 694us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.514477 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 20:24:53.515689 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.507393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:43711 (local address 127.17.109.195:33175)
0806 20:24:53.507587 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.507592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.508250 (+ 658us) server_negotiation.cc:407] Connection header received
0806 20:24:53.508458 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.508464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.508599 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.508711 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:53.509254 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.510538 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.513034 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.514326 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.514475 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.514546 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.515193 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.515366 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 20:24:53.516014 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.506424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:33207 (local address 127.17.109.194:42771)
0806 20:24:53.507050 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:53.507071 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:53.507222 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:53.507943 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:53.507949 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:53.507975 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:53.508388 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.508402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.509871 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.509877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.511694 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.511707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.513819 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.513828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.514057 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.514108 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 20:24:53.515378 (+ 1270us) client_negotiation.cc:240] Negotiation successful
0806 20:24:53.515544 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":352,"threads_started":1}
I20260806 20:24:53.517297 19610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.507221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33175 (local address 127.17.109.194:43711)
0806 20:24:53.508116 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:53.508133 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:53.508283 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:53.508723 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:53.508728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:53.508751 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:53.509092 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.509100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.510698 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.510704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.512817 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.512832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.514755 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.514761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.514943 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.514995 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 20:24:53.516841 (+ 1846us) client_negotiation.cc:240] Negotiation successful
0806 20:24:53.516987 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":537,"threads_started":1}
I20260806 20:24:53.518541 19496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c722261dedb744fe9a99f2b9c1bda456" candidate_uuid: "3abdccf167e34dd4b7a50a2e1d671386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32b96fe938f419ba4b680ad6350c8c5" is_pre_election: true
I20260806 20:24:53.519165 19496 raft_consensus.cc:2454] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3abdccf167e34dd4b7a50a2e1d671386 in term 0.
I20260806 20:24:53.521690 19275 leader_election.cc:302] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abdccf167e34dd4b7a50a2e1d671386, c32b96fe938f419ba4b680ad6350c8c5; no voters:
I20260806 20:24:53.522200 19575 raft_consensus.cc:2781] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:24:53.522360 19575 raft_consensus.cc:482] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:24:53.522468 19575 raft_consensus.cc:3037] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.524098 19204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c722261dedb744fe9a99f2b9c1bda456" candidate_uuid: "3abdccf167e34dd4b7a50a2e1d671386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c68f3a90dd43df94035de3bff64b1f" is_pre_election: true
I20260806 20:24:53.524415 19204 raft_consensus.cc:2454] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3abdccf167e34dd4b7a50a2e1d671386 in term 0.
I20260806 20:24:53.526511 19575 raft_consensus.cc:504] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.527757 19204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c722261dedb744fe9a99f2b9c1bda456" candidate_uuid: "3abdccf167e34dd4b7a50a2e1d671386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c68f3a90dd43df94035de3bff64b1f"
I20260806 20:24:53.528017 19204 raft_consensus.cc:3037] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.528018 19496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c722261dedb744fe9a99f2b9c1bda456" candidate_uuid: "3abdccf167e34dd4b7a50a2e1d671386" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32b96fe938f419ba4b680ad6350c8c5"
I20260806 20:24:53.528239 19496 raft_consensus.cc:3037] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:53.531333 19204 raft_consensus.cc:2454] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3abdccf167e34dd4b7a50a2e1d671386 in term 1.
I20260806 20:24:53.532099 19496 raft_consensus.cc:2454] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3abdccf167e34dd4b7a50a2e1d671386 in term 1.
I20260806 20:24:53.532397 19275 leader_election.cc:302] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3abdccf167e34dd4b7a50a2e1d671386, e5c68f3a90dd43df94035de3bff64b1f; no voters:
I20260806 20:24:53.532845 19575 leader_election.cc:288] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [CANDIDATE]: Term 1 election: Requested vote from peers e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207), c32b96fe938f419ba4b680ad6350c8c5 (127.17.109.195:33175)
I20260806 20:24:53.533146 19611 raft_consensus.cc:2781] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:53.533450 19611 raft_consensus.cc:686] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 LEADER]: Becoming Leader. State: Replica: 3abdccf167e34dd4b7a50a2e1d671386, State: Running, Role: LEADER
I20260806 20:24:53.533981 19611 consensus_queue.cc:237] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } }
I20260806 20:24:53.537472 19575 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: TxnStatusTablet state changed. Reason: New leader 3abdccf167e34dd4b7a50a2e1d671386. Latest consensus state: current_term: 1 leader_uuid: "3abdccf167e34dd4b7a50a2e1d671386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.537727 19575 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:24:53.538481 19613 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 20:24:53.538877 19613 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 20:24:53.539319 19030 catalog_manager.cc:5495] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3abdccf167e34dd4b7a50a2e1d671386 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "3abdccf167e34dd4b7a50a2e1d671386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } health_report { overall_health: UNKNOWN } } }
I20260806 20:24:53.591210 19604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.581923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:37207 (local address 127.0.0.1:55160)
0806 20:24:53.582561 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:53.582584 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:53.582755 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:53.583903 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:53.583909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:53.583925 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:24:53.584119 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.584131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.585834 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.585841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.587500 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.587517 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.588709 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.588715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.588963 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.589055 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:24:53.590551 (+ 1496us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:24:53.590576 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 20:24:53.590662 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 20:24:53.590840 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438}
I20260806 20:24:53.592008 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.583070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55160 (local address 127.17.109.194:37207)
0806 20:24:53.583280 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.583287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.583307 (+ 20us) server_negotiation.cc:407] Connection header received
0806 20:24:53.583409 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.583415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.583521 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.583698 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:53.584313 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.585593 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.587748 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.588428 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.588561 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.589506 (+ 945us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:53.590337 (+ 831us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:53.590488 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.591586 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.591683 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 20:24:53.598989 19204 raft_consensus.cc:1261] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Refusing update from remote peer 3abdccf167e34dd4b7a50a2e1d671386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:53.600039 19496 raft_consensus.cc:1261] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 FOLLOWER]: Refusing update from remote peer 3abdccf167e34dd4b7a50a2e1d671386: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:53.601486 19575 consensus_queue.cc:1035] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:53.602161 19575 consensus_queue.cc:1035] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:53.631053 19574 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: TxnStatusTablet state changed. Reason: New leader 3abdccf167e34dd4b7a50a2e1d671386. Latest consensus state: current_term: 1 leader_uuid: "3abdccf167e34dd4b7a50a2e1d671386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.631434 19574 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:53.633957 19611 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3abdccf167e34dd4b7a50a2e1d671386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.634310 19611 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:24:53.634526 19582 mvcc.cc:205] Tried to move back new op lower bound from 7315652172168249344 to 7315652171920060416. Current Snapshot: MvccSnapshot[applied={T|T < 7315652172168249344 or (T in {7315652172168249344})}]
I20260806 20:24:53.637887 19581 mvcc.cc:205] Tried to move back new op lower bound from 7315652172168249344 to 7315652171920060416. Current Snapshot: MvccSnapshot[applied={T|T < 7315652172168249344}]
I20260806 20:24:53.638191 19574 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3abdccf167e34dd4b7a50a2e1d671386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.638484 19574 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P c32b96fe938f419ba4b680ad6350c8c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:53.657437 19576 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: TxnStatusTablet state changed. Reason: New leader 3abdccf167e34dd4b7a50a2e1d671386. Latest consensus state: current_term: 1 leader_uuid: "3abdccf167e34dd4b7a50a2e1d671386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.657815 19576 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:53.659725 19575 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3abdccf167e34dd4b7a50a2e1d671386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.660015 19575 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P 3abdccf167e34dd4b7a50a2e1d671386: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:24:53.662709 19583 mvcc.cc:205] Tried to move back new op lower bound from 7315652172168249344 to 7315652171920060416. Current Snapshot: MvccSnapshot[applied={T|T < 7315652172168249344}]
I20260806 20:24:53.664158 19576 tablet_replica.cc:439] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3abdccf167e34dd4b7a50a2e1d671386" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 } } peers { permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 } } peers { permanent_uuid: "c32b96fe938f419ba4b680ad6350c8c5" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33175 } } }
I20260806 20:24:53.664425 19576 tablet_replica.cc:442] T c722261dedb744fe9a99f2b9c1bda456 P e5c68f3a90dd43df94035de3bff64b1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:53.708473 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.694198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:45557 (local address 127.17.109.194:37207)
0806 20:24:53.694446 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.694454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.700049 (+ 5595us) server_negotiation.cc:407] Connection header received
0806 20:24:53.700385 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.700393 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.700508 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.700608 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:53.701611 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.702887 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.705798 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.706556 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.706687 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.707549 (+ 862us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:53.707724 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:53.707869 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.708043 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.708136 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 20:24:53.708801 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.693459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:37207 (local address 127.17.109.195:45557)
0806 20:24:53.699909 (+ 6450us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:53.699942 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:53.700207 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:53.700861 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:53.700868 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:53.700882 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:24:53.701364 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.701384 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.703186 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.703192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.705551 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.705570 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.706945 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.706953 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.707194 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.707294 (+ 100us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:24:53.707869 (+ 575us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:24:53.707887 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 20:24:53.707999 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 20:24:53.708164 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4459,"thread_start_us":4268,"threads_started":1}
I20260806 20:24:53.726653 19355 raft_consensus.cc:1261] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Refusing update from remote peer c32b96fe938f419ba4b680ad6350c8c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:53.727607 19204 raft_consensus.cc:1261] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Refusing update from remote peer c32b96fe938f419ba4b680ad6350c8c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:53.727445 19590 consensus_queue.cc:1035] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3abdccf167e34dd4b7a50a2e1d671386" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:53.728309 19590 consensus_queue.cc:1035] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c68f3a90dd43df94035de3bff64b1f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 20:24:53.849933 19341 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.GetTabletLocations -> {remote=127.17.109.254:40849, user_credentials={real_user=slave}} blocked reactor thread for 48390.3us
I20260806 20:24:53.899572 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.860595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:41497 (local address 127.17.109.195:33175)
0806 20:24:53.860836 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.860843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.879094 (+ 18251us) server_negotiation.cc:407] Connection header received
0806 20:24:53.879699 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.879707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:53.879828 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:53.879903 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:53.880871 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.882070 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.896430 (+ 14360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.897425 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.897581 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.898588 (+ 1007us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:53.898768 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:53.898834 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:53.899090 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 20:24:53.899202 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 20:24:53.900730 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.861026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33175 (local address 127.17.109.194:41497)
0806 20:24:53.878977 (+ 17951us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:53.879002 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:53.879455 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:53.880137 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:53.880155 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:53.880171 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:24:53.880607 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.880626 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.882264 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.882270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.884436 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:53.884454 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:53.897856 (+ 13402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:53.897868 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:53.898203 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:53.898293 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:24:53.898942 (+ 649us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:24:53.898961 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 20:24:53.900125 (+ 1164us) client_negotiation.cc:240] Negotiation successful
0806 20:24:53.900306 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17702,"thread_start_us":18097,"threads_started":1}
I20260806 20:24:53.971918 19031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:24:53.972216 19031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:24:53.976290 19031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 87848c5e728d429c90ac459e73ed02c9: Sending DeleteTablet for 3 replicas of tablet 6797ba3d208d486aa4c55a51c2504ce1
I20260806 20:24:53.977556 19329 tablet_service.cc:1514] Processing DeleteTablet for tablet 6797ba3d208d486aa4c55a51c2504ce1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:55144
I20260806 20:24:53.978224 19474 tablet_service.cc:1514] Processing DeleteTablet for tablet 6797ba3d208d486aa4c55a51c2504ce1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:55508
I20260806 20:24:53.979084 19179 tablet_service.cc:1514] Processing DeleteTablet for tablet 6797ba3d208d486aa4c55a51c2504ce1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:36184
I20260806 20:24:53.991654 19628 tablet_replica.cc:330] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: stopping tablet replica
I20260806 20:24:53.998237 19628 raft_consensus.cc:2229] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:24:53.998669 19628 raft_consensus.cc:2258] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:53.999785 19630 tablet_replica.cc:330] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: stopping tablet replica
I20260806 20:24:54.000023 19630 raft_consensus.cc:2229] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:24:54.000208 19628 ts_tablet_manager.cc:1905] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:54.000325 19630 raft_consensus.cc:2258] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:54.000944 19629 tablet_replica.cc:330] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: stopping tablet replica
I20260806 20:24:54.001191 19629 raft_consensus.cc:2229] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:24:54.001670 19629 raft_consensus.cc:2258] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:54.002913 19630 ts_tablet_manager.cc:1905] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:54.004202 19629 ts_tablet_manager.cc:1905] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:54.009203 19628 ts_tablet_manager.cc:1918] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:24:54.009562 19628 log.cc:1196] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/6797ba3d208d486aa4c55a51c2504ce1
I20260806 20:24:54.010310 19628 ts_tablet_manager.cc:1939] T 6797ba3d208d486aa4c55a51c2504ce1 P 3abdccf167e34dd4b7a50a2e1d671386: Deleting consensus metadata
I20260806 20:24:54.011736 19018 catalog_manager.cc:4841] TS 3abdccf167e34dd4b7a50a2e1d671386 (127.17.109.194:37207): tablet 6797ba3d208d486aa4c55a51c2504ce1 (table test-table [id=2712f1f7326f463e8bc505221e4df110]) successfully deleted
I20260806 20:24:54.012585 19630 ts_tablet_manager.cc:1918] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:24:54.012770 19630 log.cc:1196] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/6797ba3d208d486aa4c55a51c2504ce1
I20260806 20:24:54.013222 19630 ts_tablet_manager.cc:1939] T 6797ba3d208d486aa4c55a51c2504ce1 P e5c68f3a90dd43df94035de3bff64b1f: Deleting consensus metadata
I20260806 20:24:54.014467 19015 catalog_manager.cc:4841] TS e5c68f3a90dd43df94035de3bff64b1f (127.17.109.193:33207): tablet 6797ba3d208d486aa4c55a51c2504ce1 (table test-table [id=2712f1f7326f463e8bc505221e4df110]) successfully deleted
Aug 06 20:24:54 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:24:54.022156 19629 ts_tablet_manager.cc:1918] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:24:54.022435 19629 log.cc:1196] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/6797ba3d208d486aa4c55a51c2504ce1
I20260806 20:24:54.023029 19629 ts_tablet_manager.cc:1939] T 6797ba3d208d486aa4c55a51c2504ce1 P c32b96fe938f419ba4b680ad6350c8c5: Deleting consensus metadata
I20260806 20:24:54.024441 19015 catalog_manager.cc:4841] TS c32b96fe938f419ba4b680ad6350c8c5 (127.17.109.195:33175): tablet 6797ba3d208d486aa4c55a51c2504ce1 (table test-table [id=2712f1f7326f463e8bc505221e4df110]) successfully deleted
I20260806 20:24:54.030866 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:53.986529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36198 (local address 127.17.109.193:33207)
0806 20:24:53.986780 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:53.986790 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:53.999548 (+ 12758us) server_negotiation.cc:407] Connection header received
0806 20:24:53.999894 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:53.999899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:54.000013 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:54.000103 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:54.002499 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.003528 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.015731 (+ 12203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.016510 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.016667 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:54.023596 (+ 6929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:54.023615 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:54.023635 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:54.023683 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:54.027416 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.028268 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.028276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.028278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.028436 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.028874 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.028878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.028881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.029374 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:54.029745 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:54.030176 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 20:24:54.030457 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260806 20:24:54.031762 19245 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36198
I20260806 20:24:54.069947 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:54.035366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.17.109.254:40849)
0806 20:24:54.050801 (+ 15435us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:54.050809 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:54.050832 (+ 23us) server_negotiation.cc:407] Connection header received
0806 20:24:54.050899 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:54.050906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:54.050991 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:54.051129 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:54.053267 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.054645 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.056886 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.057704 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.057840 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:54.061509 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:54.061516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:54.061522 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:54.061569 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:54.065942 (+ 4373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.066922 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.066928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.066932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.067129 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.067683 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.067689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.067692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.068107 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:54.068421 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:54.069280 (+ 859us) server_negotiation.cc:299] Negotiation successful
0806 20:24:54.069536 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15262,"thread_start_us":15865,"threads_started":1}
W20260806 20:24:54.070565 19031 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58792
I20260806 20:24:54.072772 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19108
I20260806 20:24:54.092383 19562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:54.074222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36208 (local address 127.17.109.193:33207)
0806 20:24:54.074434 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:54.074442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:54.076065 (+ 1623us) server_negotiation.cc:407] Connection header received
0806 20:24:54.076240 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:54.076248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:54.076345 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:54.076470 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:54.078085 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.079279 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.081185 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.081935 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.082105 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:54.085014 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:54.085024 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:54.085031 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:54.085088 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:54.089034 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.089881 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.089890 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.089895 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.090078 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.090531 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.090538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.090544 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.090983 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:54.091287 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:54.091538 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 20:24:54.091834 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 20:24:54.093158 19245 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36208
W20260806 20:24:54.093849 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19108: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:24:54.094128 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 19108
I20260806 20:24:54.119256 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19255
Aug 06 20:24:54 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:24:54.136399 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:54.120186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.17.109.194:37207)
0806 20:24:54.120410 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:54.120419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:54.120448 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:24:54.120676 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:54.120689 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:54.120758 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:54.120880 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:54.122159 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.123041 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.124587 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.125087 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.125238 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:54.129764 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:54.129777 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:54.129793 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:54.129828 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:54.133316 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.134033 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.134047 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.134050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.134222 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.134619 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.134625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.134629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.135030 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:54.135406 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:54.135736 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 20:24:54.136011 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 20:24:54.137115 19397 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55166
W20260806 20:24:54.137857 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19255: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:24:54.138018 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 19255
I20260806 20:24:54.162317 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19407
Aug 06 20:24:54 dist-test-slave-6k22 krb5kdc[18986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047892, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:24:54.181602 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:54.163214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.17.109.195:33175)
0806 20:24:54.163428 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:54.163433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:54.163454 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:24:54.163709 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:54.163714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:54.163804 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:54.163900 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:54.165282 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.165999 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.167350 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:54.167900 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:54.168010 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:54.172674 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:54.172696 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:54.172720 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:54.172779 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:54.177645 (+ 4866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.178607 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.178617 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.178621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.178916 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:54.179534 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:54.179540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:54.179544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:54.180285 (+ 741us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:54.180654 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:54.181017 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 20:24:54.181260 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260806 20:24:54.182960 19542 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55534
W20260806 20:24:54.183650 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19407: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:24:54.183846 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 19407
I20260806 20:24:54.211455 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 18999
I20260806 20:24:54.212704 19075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:24:54.330590 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 18999
2026-08-06T20:24:54Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (6391 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:24:54 dist-test-slave-6k22 krb5kdc[19640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:24:54 dist-test-slave-6k22 krb5kdc[19640](info): set up 2 sockets
Aug 06 20:24:54 dist-test-slave-6k22 krb5kdc[19640](info): commencing operation
krb5kdc: starting...
W20260806 20:24:55.501099 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:24:55 dist-test-slave-6k22 krb5kdc[19640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:24:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:24:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:24:55.715698 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46533
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:56.044752 19653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:56.045050 19653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:56.045140 19653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:56.052466 19653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:24:56.052578 19653 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 20:24:56.052619 19653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:56.052676 19653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:56.052727 19653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:24:56.052783 19653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:24:56.062803 19653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46533
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:56.064116 19653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:56.065578 19653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:56.078274 19659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:56.078222 19661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:56.078400 19658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:56.079588 19653 server_base.cc:1034] running on GCE node
I20260806 20:24:56.080494 19653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:56.081990 19653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:56.083248 19653 hybrid_clock.cc:648] HybridClock initialized: now 1786047896083185 us; error 41 us; skew 500 ppm
I20260806 20:24:56.083678 19653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:56 dist-test-slave-6k22 krb5kdc[19640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047896, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:56.092734 19653 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:24:56.094393 19653 webserver.cc:460] Webserver started at http://127.17.109.254:46073/ using document root <none> and password file <none>
I20260806 20:24:56.095178 19653 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:56.095312 19653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:56.095731 19653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:56.098842 19653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "e674a5d0b4f5492484cfe3d1c0c67a97"
format_stamp: "Formatted at 2026-08-06 20:24:56 on dist-test-slave-6k22"
server_key: "0b052500a42543971b3fa999e0eac784"
server_key_iv: "0a90195126d8c77b7c16a65913818e1f"
server_key_version: "encryptionkey@0"
I20260806 20:24:56.099722 19653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "e674a5d0b4f5492484cfe3d1c0c67a97"
format_stamp: "Formatted at 2026-08-06 20:24:56 on dist-test-slave-6k22"
server_key: "0b052500a42543971b3fa999e0eac784"
server_key_iv: "0a90195126d8c77b7c16a65913818e1f"
server_key_version: "encryptionkey@0"
I20260806 20:24:56.104987 19653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:24:56.108505 19669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:56.109881 19653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:24:56.110108 19653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "e674a5d0b4f5492484cfe3d1c0c67a97"
format_stamp: "Formatted at 2026-08-06 20:24:56 on dist-test-slave-6k22"
server_key: "0b052500a42543971b3fa999e0eac784"
server_key_iv: "0a90195126d8c77b7c16a65913818e1f"
server_key_version: "encryptionkey@0"
I20260806 20:24:56.110368 19653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:56.125744 19653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:56.130239 19653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:56.130533 19653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:56.154069 19653 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:46533
I20260806 20:24:56.154071 19732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:46533 every 8 connection(s)
I20260806 20:24:56.155628 19653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:24:56.156302 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 19653
I20260806 20:24:56.156558 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:24:56.157402 17847 external_mini_cluster.cc:1428] Setting key 212f0f2a8e0f69bd311583b3cac0edae
I20260806 20:24:56.160140 19733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 20:24:56 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:56.171018 19733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: Bootstrap starting.
I20260806 20:24:56.175470 19733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:56.176939 19733 log.cc:824] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:56.181622 19733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: No bootstrap required, opened a new log
I20260806 20:24:56.181846 19736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:56.159497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56438 (local address 127.17.109.254:46533)
0806 20:24:56.160374 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:56.160386 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:56.160424 (+ 38us) server_negotiation.cc:407] Connection header received
0806 20:24:56.161990 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:56.162011 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:56.162381 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:56.162777 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:56.164294 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.165614 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.166885 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.167600 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.167814 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:56.171939 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:56.171949 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:56.171968 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:56.172023 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:56.176408 (+ 4385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:56.177366 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:56.177377 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:56.177388 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:56.177625 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:56.178210 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:56.178257 (+ 47us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:56.178264 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:56.179164 (+ 900us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:56.179478 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:56.180120 (+ 642us) server_negotiation.cc:299] Negotiation successful
0806 20:24:56.180584 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":344,"threads_started":1}
I20260806 20:24:56.189646 19733 raft_consensus.cc:348] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674a5d0b4f5492484cfe3d1c0c67a97" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46533 } }
I20260806 20:24:56.190083 19733 raft_consensus.cc:374] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:56.190218 19733 raft_consensus.cc:729] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e674a5d0b4f5492484cfe3d1c0c67a97, State: Initialized, Role: FOLLOWER
I20260806 20:24:56.190841 19733 consensus_queue.cc:260] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674a5d0b4f5492484cfe3d1c0c67a97" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46533 } }
I20260806 20:24:56.191112 19733 raft_consensus.cc:388] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:24:56.191275 19733 raft_consensus.cc:482] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:24:56.191466 19733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:56.193629 19733 raft_consensus.cc:504] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674a5d0b4f5492484cfe3d1c0c67a97" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46533 } }
I20260806 20:24:56.194113 19733 leader_election.cc:302] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e674a5d0b4f5492484cfe3d1c0c67a97; no voters:
I20260806 20:24:56.194789 19733 leader_election.cc:288] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:24:56.194959 19738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:56.197141 19738 raft_consensus.cc:686] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [term 1 LEADER]: Becoming Leader. State: Replica: e674a5d0b4f5492484cfe3d1c0c67a97, State: Running, Role: LEADER
I20260806 20:24:56.197499 19733 sys_catalog.cc:564] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:24:56.197774 19738 consensus_queue.cc:237] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674a5d0b4f5492484cfe3d1c0c67a97" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46533 } }
I20260806 20:24:56.201143 19739 sys_catalog.cc:455] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e674a5d0b4f5492484cfe3d1c0c67a97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674a5d0b4f5492484cfe3d1c0c67a97" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46533 } } }
I20260806 20:24:56.201220 19740 sys_catalog.cc:455] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e674a5d0b4f5492484cfe3d1c0c67a97. Latest consensus state: current_term: 1 leader_uuid: "e674a5d0b4f5492484cfe3d1c0c67a97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674a5d0b4f5492484cfe3d1c0c67a97" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46533 } } }
I20260806 20:24:56.201505 19739 sys_catalog.cc:458] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [sys.catalog]: This master's current role is: LEADER
I20260806 20:24:56.201608 19740 sys_catalog.cc:458] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97 [sys.catalog]: This master's current role is: LEADER
I20260806 20:24:56.203408 19745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:24:56.209210 19745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:24:56.218901 19745 catalog_manager.cc:1347] Generated new cluster ID: 6c4e8fccde6948ce97664198149ebd22
I20260806 20:24:56.219028 19745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:24:56.233429 19745 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:24:56.235630 19745 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:24:56.249126 19745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: Generated new TSK 0
I20260806 20:24:56.250227 19745 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:24:56.301539 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46533
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:56.635713 19759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:56.636052 19759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:56.636234 19759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:56.643512 19759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:56.643705 19759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:56.643853 19759 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 20:24:56.643941 19759 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:24:56.644076 19759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:24:56.654384 19759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:46533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:56.655910 19759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:56.657292 19759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:56.670652 19767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:56.670697 19765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:56.670619 19764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:56.672288 19759 server_base.cc:1034] running on GCE node
I20260806 20:24:56.673308 19759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:56.674392 19759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:56.675695 19759 hybrid_clock.cc:648] HybridClock initialized: now 1786047896675635 us; error 47 us; skew 500 ppm
I20260806 20:24:56.676064 19759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:56 dist-test-slave-6k22 krb5kdc[19640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047896, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:56.679519 19759 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:24:56.681258 19759 webserver.cc:460] Webserver started at http://127.17.109.193:33805/ using document root <none> and password file <none>
I20260806 20:24:56.682003 19759 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:56.682163 19759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:56.682487 19759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:56.685617 19759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "10fc8c091aaf4b5cbe02340b0a2fd243"
format_stamp: "Formatted at 2026-08-06 20:24:56 on dist-test-slave-6k22"
server_key: "1be467aab96c8380eba1d41d1d1caff6"
server_key_iv: "2bec2baa7deb166a1ca075e90a876872"
server_key_version: "encryptionkey@0"
I20260806 20:24:56.686411 19759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "10fc8c091aaf4b5cbe02340b0a2fd243"
format_stamp: "Formatted at 2026-08-06 20:24:56 on dist-test-slave-6k22"
server_key: "1be467aab96c8380eba1d41d1d1caff6"
server_key_iv: "2bec2baa7deb166a1ca075e90a876872"
server_key_version: "encryptionkey@0"
I20260806 20:24:56.691910 19759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:24:56.695521 19775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:56.697043 19759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:24:56.697320 19759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "10fc8c091aaf4b5cbe02340b0a2fd243"
format_stamp: "Formatted at 2026-08-06 20:24:56 on dist-test-slave-6k22"
server_key: "1be467aab96c8380eba1d41d1d1caff6"
server_key_iv: "2bec2baa7deb166a1ca075e90a876872"
server_key_version: "encryptionkey@0"
I20260806 20:24:56.697587 19759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:56.724512 19759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:56.729089 19759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:56.729427 19759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:56.732566 19759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:56.732733 19759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:56.732888 19759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:56.732995 19759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:24:56 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047896, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:56.761299 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:56.732868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:45257 (local address 127.17.109.254:46533)
0806 20:24:56.733491 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:56.733498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:56.734712 (+ 1214us) server_negotiation.cc:407] Connection header received
0806 20:24:56.738126 (+ 3414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:56.738131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:56.738223 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:56.738403 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:56.741206 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.742772 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.745384 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.746236 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.746434 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:56.752201 (+ 5767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:56.752211 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:56.752219 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:56.752276 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:56.756406 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:56.757723 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:56.757732 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:56.757738 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:56.757938 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:56.758618 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:56.758626 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:56.758632 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:56.759080 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:56.759251 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:56.760446 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0806 20:24:56.760762 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":339,"threads_started":1}
I20260806 20:24:56.763116 19786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:56.733589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46533 (local address 127.17.109.193:45257)
0806 20:24:56.734492 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:56.734556 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:56.737733 (+ 3177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:56.738743 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:56.738759 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:56.739744 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:56.740938 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:56.740979 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.742965 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.742974 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:56.745209 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:56.745222 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.746597 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.746604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:56.746831 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:56.747701 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:56.747739 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:56.751861 (+ 4122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:56.756608 (+ 4747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:56.756622 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:56.756645 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:56.757530 (+ 885us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:56.758085 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:56.758092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:56.758097 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:56.758460 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:56.759220 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:56.759226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:56.759897 (+ 671us) client_negotiation.cc:769] Sending connection context
0806 20:24:56.760301 (+ 404us) client_negotiation.cc:240] Negotiation successful
0806 20:24:56.761036 (+ 735us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":345,"threads_started":1}
I20260806 20:24:56.775759 19759 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:39647
I20260806 20:24:56.775822 19902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:39647 every 8 connection(s)
I20260806 20:24:56.777184 19759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:24:56.779577 19903 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46533
I20260806 20:24:56.779948 19903 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:56.780787 19903 heartbeater.cc:511] Master 127.17.109.254:46533 requested a full tablet report, sending...
I20260806 20:24:56.783326 19686 ts_manager.cc:194] Registered new tserver with Master: 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647)
I20260806 20:24:56.784399 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 19759
I20260806 20:24:56.784644 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:24:56.785286 19686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:45257
I20260806 20:24:56.785537 17847 external_mini_cluster.cc:1428] Setting key 31ce4d809346a9aac18bfe37373685dc
I20260806 20:24:56.787775 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:56.772403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:34251 (local address 127.17.109.254:46533)
0806 20:24:56.772643 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:56.772651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:56.772980 (+ 329us) server_negotiation.cc:407] Connection header received
0806 20:24:56.773130 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:56.773139 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:56.773237 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:56.773358 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:56.774594 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.775881 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.777310 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.778094 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.778264 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:56.780741 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:56.780746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:56.780749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:56.780781 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:56.784395 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:56.785327 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:56.785334 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:56.785337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:56.785493 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:56.785966 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:56.785970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:56.785973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:56.786331 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:56.786553 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:56.786998 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 20:24:56.787226 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 20:24:56.788028 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:56.771875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46533 (local address 127.17.109.193:34251)
0806 20:24:56.772864 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:56.772892 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:56.773010 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:56.773407 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:56.773418 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:56.774129 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:56.774426 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:56.774436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.776025 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.776030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:56.777124 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:56.777137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:56.778307 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:56.778314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:56.778580 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:56.779360 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:56.779384 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:56.780556 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:56.784626 (+ 4070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:56.784634 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:56.784640 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:56.785191 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:56.785616 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:56.785622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:56.785625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:56.785836 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:56.786513 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:56.786518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:56.786721 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 20:24:56.786871 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 20:24:56.787211 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":683,"threads_started":1}
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:24:56.835249 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46533
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:57.183192 19907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:57.183512 19907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:57.183720 19907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:57.191927 19907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:57.192029 19907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:57.192121 19907 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 20:24:57.192179 19907 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:24:57.192240 19907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:24:57.203480 19907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:46533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:57.204938 19907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:57.206260 19907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:57.219725 19912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:57.219764 19913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:57.219806 19915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:57.220805 19907 server_base.cc:1034] running on GCE node
I20260806 20:24:57.221460 19907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:57.222406 19907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:57.223731 19907 hybrid_clock.cc:648] HybridClock initialized: now 1786047897223638 us; error 46 us; skew 500 ppm
I20260806 20:24:57.224062 19907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:57 dist-test-slave-6k22 krb5kdc[19640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047897, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:57.227281 19907 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:24:57.228829 19907 webserver.cc:460] Webserver started at http://127.17.109.194:33047/ using document root <none> and password file <none>
I20260806 20:24:57.229487 19907 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:57.229591 19907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:57.229864 19907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:57.232801 19907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "5a3b81068bc342a4b8362ca5aa0dcee9"
format_stamp: "Formatted at 2026-08-06 20:24:57 on dist-test-slave-6k22"
server_key: "877efd8de434a597723e421d001d3c0c"
server_key_iv: "6396ac95ed0c2749c95a8cde71f6f567"
server_key_version: "encryptionkey@0"
I20260806 20:24:57.233513 19907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "5a3b81068bc342a4b8362ca5aa0dcee9"
format_stamp: "Formatted at 2026-08-06 20:24:57 on dist-test-slave-6k22"
server_key: "877efd8de434a597723e421d001d3c0c"
server_key_iv: "6396ac95ed0c2749c95a8cde71f6f567"
server_key_version: "encryptionkey@0"
I20260806 20:24:57.238639 19907 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:24:57.242128 19922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:57.243660 19907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:24:57.243849 19907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "5a3b81068bc342a4b8362ca5aa0dcee9"
format_stamp: "Formatted at 2026-08-06 20:24:57 on dist-test-slave-6k22"
server_key: "877efd8de434a597723e421d001d3c0c"
server_key_iv: "6396ac95ed0c2749c95a8cde71f6f567"
server_key_version: "encryptionkey@0"
I20260806 20:24:57.244055 19907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:57.259305 19907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:57.263967 19907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:57.264292 19907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:57.267347 19907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:57.267496 19907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:57.267679 19907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:57.267817 19907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:24:57 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047897, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:57.290231 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.267622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:34157 (local address 127.17.109.254:46533)
0806 20:24:57.267817 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:57.267823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:57.269205 (+ 1382us) server_negotiation.cc:407] Connection header received
0806 20:24:57.271277 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:57.271286 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:57.271406 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:57.271591 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:57.273448 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.274548 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.276934 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.277489 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.277616 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.282881 (+ 5265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:57.282887 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:57.282890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:57.282924 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:57.286563 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.287537 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.287542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.287546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.287715 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.288152 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.288157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.288161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.288462 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:57.288629 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:57.289806 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0806 20:24:57.289970 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 20:24:57.292613 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.268163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46533 (local address 127.17.109.194:34157)
0806 20:24:57.269011 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:57.269065 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:57.270978 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:57.271869 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:57.271876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:57.272399 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:57.273199 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.273220 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.274778 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.274783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.276742 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.276753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.278019 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.278024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.278344 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.279218 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:57.279240 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:57.282596 (+ 3356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:57.286702 (+ 4106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:57.286711 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:57.286727 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:57.287399 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:57.287822 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:57.287827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:57.287831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:57.288032 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:57.288639 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:57.288653 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:57.289254 (+ 601us) client_negotiation.cc:769] Sending connection context
0806 20:24:57.289672 (+ 418us) client_negotiation.cc:240] Negotiation successful
0806 20:24:57.290430 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":385,"threads_started":1}
I20260806 20:24:57.322510 19907 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:44735
I20260806 20:24:57.322572 20048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:44735 every 8 connection(s)
I20260806 20:24:57.323988 19907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:24:57.326357 20049 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46533
I20260806 20:24:57.326632 20049 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:57.327512 20049 heartbeater.cc:511] Master 127.17.109.254:46533 requested a full tablet report, sending...
I20260806 20:24:57.328892 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.313860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:52247 (local address 127.17.109.254:46533)
0806 20:24:57.314174 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:57.314185 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:57.314716 (+ 531us) server_negotiation.cc:407] Connection header received
0806 20:24:57.314886 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:57.314895 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:57.315014 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:57.315182 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:57.316447 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.317827 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.319529 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.320329 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.320474 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.322577 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:57.322583 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:57.322588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:57.322644 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:57.325803 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.326698 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.326705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.326711 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.326891 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.327263 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.327267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.327270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.327780 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:57.327941 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:57.328325 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 20:24:57.328523 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:24:57.329056 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.313845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46533 (local address 127.17.109.194:52247)
0806 20:24:57.314599 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:57.314617 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:57.314745 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:57.315211 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:57.315216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:57.315746 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:57.316247 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.316264 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.318043 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.318055 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.319359 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.319371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.320513 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.320517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.320688 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.321410 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:57.321440 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:57.322377 (+ 937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:57.325967 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:57.325973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:57.325976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:57.326548 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:57.327007 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:57.327011 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:57.327013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:57.327148 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:57.327900 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:57.327904 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:57.328067 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 20:24:57.328211 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 20:24:57.328523 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":423,"threads_started":1}
I20260806 20:24:57.329155 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 19907
I20260806 20:24:57.329348 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:24:57.329877 17847 external_mini_cluster.cc:1428] Setting key ad54d7a7ce1e8fbd581468372a371626
I20260806 20:24:57.330847 19685 ts_manager.cc:194] Registered new tserver with Master: 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735)
I20260806 20:24:57.332090 19685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:34157
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:24:57.378736 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46533
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:42989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:24:57.713529 20053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:24:57.713819 20053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:24:57.713997 20053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:24:57.721621 20053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:24:57.721732 20053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:24:57.721911 20053 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 20:24:57.721974 20053 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:24:57.722081 20053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:24:57.732298 20053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:46533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:24:57.733569 20053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:24:57.734905 20053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:24:57.747990 20058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:57.748044 20059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:24:57.748070 20061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:24:57.749015 20053 server_base.cc:1034] running on GCE node
I20260806 20:24:57.749579 20053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:24:57.750450 20053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:24:57.751669 20053 hybrid_clock.cc:648] HybridClock initialized: now 1786047897751601 us; error 48 us; skew 500 ppm
I20260806 20:24:57.751999 20053 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:24:57 dist-test-slave-6k22 krb5kdc[19640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047897, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:24:57.755381 20053 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:24:57.756898 20053 webserver.cc:460] Webserver started at http://127.17.109.195:35097/ using document root <none> and password file <none>
I20260806 20:24:57.757589 20053 fs_manager.cc:359] Metadata directory not provided
I20260806 20:24:57.757725 20053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:24:57.758072 20053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:24:57.761130 20053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "faa098785cd3413daed70a1a0fadf549"
format_stamp: "Formatted at 2026-08-06 20:24:57 on dist-test-slave-6k22"
server_key: "b18f3ad5534a7445c795e3ce82493dd5"
server_key_iv: "e5624102ad004da4ea883394466b108d"
server_key_version: "encryptionkey@0"
I20260806 20:24:57.761895 20053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "faa098785cd3413daed70a1a0fadf549"
format_stamp: "Formatted at 2026-08-06 20:24:57 on dist-test-slave-6k22"
server_key: "b18f3ad5534a7445c795e3ce82493dd5"
server_key_iv: "e5624102ad004da4ea883394466b108d"
server_key_version: "encryptionkey@0"
I20260806 20:24:57.767429 20053 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 20:24:57.771097 20070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:57.772761 20053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:24:57.772958 20053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "faa098785cd3413daed70a1a0fadf549"
format_stamp: "Formatted at 2026-08-06 20:24:57 on dist-test-slave-6k22"
server_key: "b18f3ad5534a7445c795e3ce82493dd5"
server_key_iv: "e5624102ad004da4ea883394466b108d"
server_key_version: "encryptionkey@0"
I20260806 20:24:57.773185 20053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:24:57.789156 19903 heartbeater.cc:503] Master 127.17.109.254:46533 was elected leader, sending a full tablet report...
I20260806 20:24:57.790582 20053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:24:57.795516 20053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:24:57.795893 20053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:24:57.798714 20053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:24:57.798851 20053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:24:57.798975 20053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:24:57.799069 20053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:24:57 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047897, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:57.826417 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.799606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:38241 (local address 127.17.109.254:46533)
0806 20:24:57.799869 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:57.799873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:57.801294 (+ 1421us) server_negotiation.cc:407] Connection header received
0806 20:24:57.803980 (+ 2686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:57.803986 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:57.804094 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:57.804202 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:57.806913 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.808363 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.811495 (+ 3132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.812355 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.812507 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.818319 (+ 5812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:57.818327 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:57.818335 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:57.818392 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:57.822395 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.823330 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.823338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.823344 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.823533 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.824069 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.824077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.824083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.824545 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:57.824735 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:57.825509 (+ 774us) server_negotiation.cc:299] Negotiation successful
0806 20:24:57.825890 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260806 20:24:57.827003 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.800200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46533 (local address 127.17.109.195:38241)
0806 20:24:57.801127 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:57.801175 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:57.803623 (+ 2448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:57.804541 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:57.804558 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:57.805570 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:57.806656 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.806691 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.808594 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.808599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.811281 (+ 2682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.811302 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.812667 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.812678 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.813086 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.814051 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:57.814090 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:57.818032 (+ 3942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:57.822586 (+ 4554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:57.822596 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:57.822610 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:57.823177 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:57.823702 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:57.823707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:57.823710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:57.823951 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:57.824694 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:57.824700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:57.825044 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 20:24:57.825290 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 20:24:57.825697 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":357,"threads_started":1}
I20260806 20:24:57.846303 20053 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:34375
I20260806 20:24:57.846346 20197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:34375 every 8 connection(s)
I20260806 20:24:57.847751 20053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:24:57.850436 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.833750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46533 (local address 127.17.109.195:50023)
0806 20:24:57.836261 (+ 2511us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:57.836276 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:57.836448 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:57.836832 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:57.836837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:57.837469 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:57.837968 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.837983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.839142 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.839146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.840831 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.840857 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.842054 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.842058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.842176 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.842911 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:57.842937 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:57.844287 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:57.847624 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:57.847632 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:57.847639 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:57.848184 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:57.848732 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:57.848737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:57.848740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:57.848908 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:57.849604 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:57.849608 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:57.849761 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 20:24:57.849869 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 20:24:57.850094 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2378,"thread_start_us":2277,"threads_started":1}
I20260806 20:24:57.850690 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.834204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:50023 (local address 127.17.109.254:46533)
0806 20:24:57.834391 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:57.834396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:57.836365 (+ 1969us) server_negotiation.cc:407] Connection header received
0806 20:24:57.836597 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:57.836602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:57.836696 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:57.836807 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:57.838126 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.838946 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.841035 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.841858 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.842021 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.844487 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:57.844492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:57.844495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:57.844526 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:57.847452 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.848341 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.848349 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.848354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.848527 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.849066 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.849070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.849073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.849439 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:57.849646 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:57.850046 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 20:24:57.850287 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 20:24:57.851142 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 20053
I20260806 20:24:57.851346 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:24:57.851655 20198 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46533
I20260806 20:24:57.851904 20198 heartbeater.cc:464] Registering TS with master...
I20260806 20:24:57.851979 17847 external_mini_cluster.cc:1428] Setting key 9ba510ff79605e6fedbfc9e4a86317ff
I20260806 20:24:57.852829 20198 heartbeater.cc:511] Master 127.17.109.254:46533 requested a full tablet report, sending...
I20260806 20:24:57.854423 19685 ts_manager.cc:194] Registered new tserver with Master: faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375)
I20260806 20:24:57.855409 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:24:57.855418 19685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:38241
Aug 06 20:24:57 dist-test-slave-6k22 krb5kdc[19640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 20:24:57 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:57.908962 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.893768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56442 (local address 127.17.109.254:46533)
0806 20:24:57.894009 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:57.894016 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:57.894352 (+ 336us) server_negotiation.cc:407] Connection header received
0806 20:24:57.894520 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:57.894524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:57.894597 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:57.894706 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:57.896240 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.897108 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.898406 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.898958 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.899073 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.903137 (+ 4064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:57.903142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:57.903145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:57.903202 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:57.906175 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.906816 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.906820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.906823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.906945 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:57.907288 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:57.907291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:57.907293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:57.907854 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:57.908048 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:57.908417 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 20:24:57.908610 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 20:24:57.913420 19685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:24:57.921185 19685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:24:57.962924 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.935605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.17.109.193:39647)
0806 20:24:57.943807 (+ 8202us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:57.943821 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:57.953149 (+ 9328us) server_negotiation.cc:407] Connection header received
0806 20:24:57.953531 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:57.953547 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:57.953913 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:57.954140 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:57.955298 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.956818 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.960222 (+ 3404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.961595 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.961830 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.962033 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:57.962348 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 20:24:57.962544 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8015,"thread_start_us":7753,"threads_started":1}
I20260806 20:24:57.963435 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.934858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:39647 (local address 127.0.0.1:52380)
0806 20:24:57.952986 (+ 18128us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:57.953013 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:57.953190 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:57.954515 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:57.954523 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:57.954555 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:57.955017 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.955034 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.956964 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.956970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.959990 (+ 3020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.960012 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.961846 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.961853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.962073 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.962132 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 20:24:57.962298 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 20:24:57.962544 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18019,"mutex_wait_us":37,"thread_start_us":18299,"threads_started":1}
I20260806 20:24:57.963804 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.934466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50646 (local address 127.17.109.195:34375)
0806 20:24:57.954098 (+ 19632us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:57.954110 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:57.954147 (+ 37us) server_negotiation.cc:407] Connection header received
0806 20:24:57.954262 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:57.954272 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:57.954544 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:57.954779 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:57.956119 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.957902 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.960273 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.961819 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.962031 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.962256 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:57.963221 (+ 965us) server_negotiation.cc:299] Negotiation successful
0806 20:24:57.963419 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19420,"thread_start_us":19196,"threads_started":1}
I20260806 20:24:57.964066 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.934219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:34375 (local address 127.0.0.1:50646)
0806 20:24:57.945346 (+ 11127us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:57.945410 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:57.945640 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:57.955134 (+ 9494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:57.955142 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:57.955172 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:57.955841 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.955858 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.958066 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.958075 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.960052 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.960065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.962423 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.962430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.962616 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.962652 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 20:24:57.962746 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 20:24:57.962900 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10737,"thread_start_us":11682,"threads_started":1}
I20260806 20:24:57.964701 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.935045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.17.109.194:44735)
0806 20:24:57.954588 (+ 19543us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:57.954600 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:57.954630 (+ 30us) server_negotiation.cc:407] Connection header received
0806 20:24:57.954738 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:57.954747 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:57.954981 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:57.955837 (+ 856us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:57.956093 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.957745 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.960998 (+ 3253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.962629 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.962830 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.962991 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:57.964198 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0806 20:24:57.964395 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19357,"thread_start_us":20101,"threads_started":1}
I20260806 20:24:57.964810 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:57.934473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44735 (local address 127.0.0.1:49208)
0806 20:24:57.952784 (+ 18311us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:57.952833 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:57.953039 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:57.955233 (+ 2194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:57.955240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:57.955271 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:57.955919 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.955936 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.957918 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.957924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.960804 (+ 2880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:57.960820 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:57.963441 (+ 2621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:57.963451 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:57.963708 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:57.963748 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 20:24:57.963844 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 20:24:57.963993 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18077,"mutex_wait_us":105,"thread_start_us":17853,"threads_started":1}
I20260806 20:24:57.968039 19828 tablet_service.cc:1467] Processing CreateTablet for tablet c1cd4b0f99ef48eaabc55a45309e47e0 (DEFAULT_TABLE table=test-table [id=966b069e9f764d489c152062d7259883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:57.969373 19828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1cd4b0f99ef48eaabc55a45309e47e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:57.968847 20123 tablet_service.cc:1467] Processing CreateTablet for tablet c1cd4b0f99ef48eaabc55a45309e47e0 (DEFAULT_TABLE table=test-table [id=966b069e9f764d489c152062d7259883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:57.968850 19975 tablet_service.cc:1467] Processing CreateTablet for tablet c1cd4b0f99ef48eaabc55a45309e47e0 (DEFAULT_TABLE table=test-table [id=966b069e9f764d489c152062d7259883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:57.970327 19975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1cd4b0f99ef48eaabc55a45309e47e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:57.970319 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1cd4b0f99ef48eaabc55a45309e47e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:57.982436 20217 tablet_bootstrap.cc:492] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Bootstrap starting.
I20260806 20:24:57.985458 20219 tablet_bootstrap.cc:492] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Bootstrap starting.
I20260806 20:24:57.986836 20217 tablet_bootstrap.cc:654] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:57.987265 20218 tablet_bootstrap.cc:492] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Bootstrap starting.
I20260806 20:24:57.989382 20217 log.cc:824] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:57.989473 20219 tablet_bootstrap.cc:654] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:57.990757 20218 tablet_bootstrap.cc:654] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:57.990984 20219 log.cc:824] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:57.992077 20218 log.cc:824] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Log is configured to *not* fsync() on all Append() calls
I20260806 20:24:57.993340 20217 tablet_bootstrap.cc:492] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: No bootstrap required, opened a new log
I20260806 20:24:57.993661 20217 ts_tablet_manager.cc:1397] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260806 20:24:57.994377 20219 tablet_bootstrap.cc:492] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: No bootstrap required, opened a new log
I20260806 20:24:57.994696 20218 tablet_bootstrap.cc:492] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: No bootstrap required, opened a new log
I20260806 20:24:57.994755 20219 ts_tablet_manager.cc:1397] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260806 20:24:57.994966 20218 ts_tablet_manager.cc:1397] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260806 20:24:58.003257 20218 raft_consensus.cc:348] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.003681 20218 raft_consensus.cc:374] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.003789 20218 raft_consensus.cc:729] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3b81068bc342a4b8362ca5aa0dcee9, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.004513 20218 consensus_queue.cc:260] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.005681 20049 heartbeater.cc:503] Master 127.17.109.254:46533 was elected leader, sending a full tablet report...
I20260806 20:24:58.006209 20218 ts_tablet_manager.cc:1428] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260806 20:24:58.006520 20217 raft_consensus.cc:348] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.006984 20217 raft_consensus.cc:374] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.007090 20217 raft_consensus.cc:729] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fc8c091aaf4b5cbe02340b0a2fd243, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.007784 20219 raft_consensus.cc:348] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.008250 20219 raft_consensus.cc:374] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.008370 20219 raft_consensus.cc:729] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa098785cd3413daed70a1a0fadf549, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.008555 20217 consensus_queue.cc:260] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.009017 20219 consensus_queue.cc:260] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.010481 20217 ts_tablet_manager.cc:1428] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent starting tablet: real 0.017s user 0.007s sys 0.009s
I20260806 20:24:58.011524 20198 heartbeater.cc:503] Master 127.17.109.254:46533 was elected leader, sending a full tablet report...
I20260806 20:24:58.012384 20219 ts_tablet_manager.cc:1428] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Time spent starting tablet: real 0.017s user 0.014s sys 0.005s
W20260806 20:24:58.032115 19904 tablet.cc:2369] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:24:58.046065 20225 raft_consensus.cc:482] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:24:58.046447 20225 raft_consensus.cc:504] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.048472 20225 leader_election.cc:288] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375), 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735)
I20260806 20:24:58.057050 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.049305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:49051 (local address 127.17.109.194:44735)
0806 20:24:58.049572 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.049581 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.050341 (+ 760us) server_negotiation.cc:407] Connection header received
0806 20:24:58.050682 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.050691 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.050771 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.050840 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:58.051657 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.052449 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.055054 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.055921 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.056074 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.056175 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.056643 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.056771 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 20:24:58.057063 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.048965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:59653 (local address 127.17.109.195:34375)
0806 20:24:58.049199 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.049206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.050539 (+ 1333us) server_negotiation.cc:407] Connection header received
0806 20:24:58.050768 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.050772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.050969 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.051065 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:58.051760 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.052479 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.054979 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.055960 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.056185 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.056271 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.056611 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.056757 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 20:24:58.057469 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.048965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:34375 (local address 127.17.109.193:59653)
0806 20:24:58.050428 (+ 1463us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.050452 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.050594 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.051094 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.051097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.051126 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:58.051535 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.051593 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.052678 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.052682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.054806 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.054822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.056239 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.056244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.056354 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.056460 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.056541 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.056698 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1343,"mutex_wait_us":269,"spinlock_wait_cycles":6912,"thread_start_us":638,"threads_started":1}
I20260806 20:24:58.057596 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.048946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44735 (local address 127.17.109.193:49051)
0806 20:24:58.050226 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.050247 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.050473 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.050990 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.050996 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.051028 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:58.051475 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.051486 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.052587 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.052596 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.054870 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.054905 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.056190 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.056198 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.056450 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.056476 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.056602 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.056757 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1065,"mutex_wait_us":101,"thread_start_us":1147,"threads_started":1}
I20260806 20:24:58.058643 19997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1cd4b0f99ef48eaabc55a45309e47e0" candidate_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" is_pre_election: true
I20260806 20:24:58.058646 20145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1cd4b0f99ef48eaabc55a45309e47e0" candidate_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa098785cd3413daed70a1a0fadf549" is_pre_election: true
I20260806 20:24:58.059177 19997 raft_consensus.cc:2454] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10fc8c091aaf4b5cbe02340b0a2fd243 in term 0.
I20260806 20:24:58.059206 20145 raft_consensus.cc:2454] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10fc8c091aaf4b5cbe02340b0a2fd243 in term 0.
I20260806 20:24:58.059924 19776 leader_election.cc:302] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10fc8c091aaf4b5cbe02340b0a2fd243, 5a3b81068bc342a4b8362ca5aa0dcee9, faa098785cd3413daed70a1a0fadf549; no voters:
I20260806 20:24:58.060365 20225 raft_consensus.cc:2781] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:24:58.060554 20225 raft_consensus.cc:482] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:24:58.060740 20225 raft_consensus.cc:3037] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.063037 20225 raft_consensus.cc:504] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.063807 20225 leader_election.cc:288] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [CANDIDATE]: Term 1 election: Requested vote from peers faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375), 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735)
I20260806 20:24:58.064307 20145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1cd4b0f99ef48eaabc55a45309e47e0" candidate_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa098785cd3413daed70a1a0fadf549"
I20260806 20:24:58.064386 19997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1cd4b0f99ef48eaabc55a45309e47e0" candidate_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9"
I20260806 20:24:58.064548 20145 raft_consensus.cc:3037] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.064580 19997 raft_consensus.cc:3037] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.068022 19997 raft_consensus.cc:2454] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10fc8c091aaf4b5cbe02340b0a2fd243 in term 1.
I20260806 20:24:58.068022 20145 raft_consensus.cc:2454] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10fc8c091aaf4b5cbe02340b0a2fd243 in term 1.
I20260806 20:24:58.068570 19776 leader_election.cc:302] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10fc8c091aaf4b5cbe02340b0a2fd243, 5a3b81068bc342a4b8362ca5aa0dcee9, faa098785cd3413daed70a1a0fadf549; no voters:
I20260806 20:24:58.068981 20225 raft_consensus.cc:2781] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:58.069496 20225 raft_consensus.cc:686] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 LEADER]: Becoming Leader. State: Replica: 10fc8c091aaf4b5cbe02340b0a2fd243, State: Running, Role: LEADER
I20260806 20:24:58.070043 20225 consensus_queue.cc:237] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } }
I20260806 20:24:58.075186 19686 catalog_manager.cc:5495] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193). New cstate: current_term: 1 leader_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } health_report { overall_health: HEALTHY } } }
W20260806 20:24:58.076330 20050 tablet.cc:2369] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:24:58.099953 20199 tablet.cc:2369] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 20:24:58 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047896, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:24:58.149091 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.134453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46533 (local address 127.0.0.1:56446)
0806 20:24:58.135186 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.135204 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.135393 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.135823 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.135828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.136625 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:24:58.136911 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.136921 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.138014 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.138022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.139118 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.139131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.139996 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.140003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.140191 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.140863 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:24:58.140882 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:24:58.143185 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:24:58.146275 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:58.146290 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:58.146299 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:58.146750 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:58.147160 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:24:58.147165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:24:58.147171 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:24:58.147367 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:24:58.148068 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:24:58.148076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:24:58.148254 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.148443 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.148746 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":422,"threads_started":1}
I20260806 20:24:58.149381 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.134438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.17.109.254:46533)
0806 20:24:58.134637 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.134711 (+ 74us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.135302 (+ 591us) server_negotiation.cc:407] Connection header received
0806 20:24:58.135521 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.135533 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.135673 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.135788 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:24:58.137082 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.137880 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.139275 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.139800 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.139950 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.143378 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:24:58.143383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:24:58.143387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:24:58.143416 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:24:58.146137 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:58.146874 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:58.146882 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:58.146886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:58.147028 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:24:58.147497 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:24:58.147504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:24:58.147512 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:24:58.147903 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:24:58.148076 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.148550 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.148967 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":70}
I20260806 20:24:58.154433 19686 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:56446:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 20:24:58.155793 19686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:24:58.164216 20123 tablet_service.cc:1467] Processing CreateTablet for tablet ed0c8d279994488f8dec8b7e0b146f37 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56148585417948c4a70fb9b71599d045]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:24:58.164149 19828 tablet_service.cc:1467] Processing CreateTablet for tablet ed0c8d279994488f8dec8b7e0b146f37 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56148585417948c4a70fb9b71599d045]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:24:58.165164 19828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0c8d279994488f8dec8b7e0b146f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:58.165195 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0c8d279994488f8dec8b7e0b146f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:58.171000 20219 tablet_bootstrap.cc:492] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: Bootstrap starting.
I20260806 20:24:58.172573 20219 tablet_bootstrap.cc:654] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:58.173251 19975 tablet_service.cc:1467] Processing CreateTablet for tablet ed0c8d279994488f8dec8b7e0b146f37 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56148585417948c4a70fb9b71599d045]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:24:58.174252 19975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed0c8d279994488f8dec8b7e0b146f37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:58.178318 20217 tablet_bootstrap.cc:492] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: Bootstrap starting.
I20260806 20:24:58.180591 20219 tablet_bootstrap.cc:492] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: No bootstrap required, opened a new log
I20260806 20:24:58.180794 20219 ts_tablet_manager.cc:1397] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 20:24:58.181769 20219 raft_consensus.cc:348] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.182060 20219 raft_consensus.cc:374] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.182168 20219 raft_consensus.cc:729] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa098785cd3413daed70a1a0fadf549, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.182857 20218 tablet_bootstrap.cc:492] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: Bootstrap starting.
I20260806 20:24:58.183418 20217 tablet_bootstrap.cc:654] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:58.182497 20219 consensus_queue.cc:260] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.184615 20219 ts_tablet_manager.cc:1428] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 20:24:58.184856 20218 tablet_bootstrap.cc:654] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:58.185873 20226 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.186285 20226 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:58.187286 20217 tablet_bootstrap.cc:492] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: No bootstrap required, opened a new log
I20260806 20:24:58.187461 20217 ts_tablet_manager.cc:1397] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.001s
I20260806 20:24:58.188361 20217 raft_consensus.cc:348] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.188638 20217 raft_consensus.cc:374] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.188745 20217 raft_consensus.cc:729] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fc8c091aaf4b5cbe02340b0a2fd243, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.188997 20217 consensus_queue.cc:260] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.189652 20217 ts_tablet_manager.cc:1428] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 20:24:58.189456 20225 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.190392 20225 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:58.191452 20218 tablet_bootstrap.cc:492] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: No bootstrap required, opened a new log
I20260806 20:24:58.191679 20218 ts_tablet_manager.cc:1397] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 20:24:58.193091 20218 raft_consensus.cc:348] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.193368 20218 raft_consensus.cc:374] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.193476 20218 raft_consensus.cc:729] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3b81068bc342a4b8362ca5aa0dcee9, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.193732 20218 consensus_queue.cc:260] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.194401 20218 ts_tablet_manager.cc:1428] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260806 20:24:58.194209 20224 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.194690 20224 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:58.412441 20226 raft_consensus.cc:482] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:24:58.412740 20226 raft_consensus.cc:504] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.414567 20226 leader_election.cc:288] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647), 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735)
I20260806 20:24:58.424337 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.415096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:38563 (local address 127.17.109.194:44735)
0806 20:24:58.415383 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.415393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.416040 (+ 647us) server_negotiation.cc:407] Connection header received
0806 20:24:58.416425 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.416431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.416511 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.416577 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:58.417317 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.418633 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.421545 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.423014 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.423198 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.423282 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.423893 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.424037 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 20:24:58.424317 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.415200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:55533 (local address 127.17.109.193:39647)
0806 20:24:58.415491 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.415501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.416106 (+ 605us) server_negotiation.cc:407] Connection header received
0806 20:24:58.416439 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.416446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.416526 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.416595 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:58.417477 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.418571 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.421545 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.423014 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.423212 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.423282 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.423893 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.424037 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 20:24:58.424548 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.414912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:39647 (local address 127.17.109.195:55533)
0806 20:24:58.415987 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.416007 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.416237 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.416826 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.416834 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.416897 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:58.417315 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.417325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.418734 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.418738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.421348 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.421364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.423288 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.423294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.423604 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.423751 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.423841 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.424038 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":239,"spinlock_wait_cycles":163200,"thread_start_us":692,"threads_started":1}
I20260806 20:24:58.424609 20241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.414839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44735 (local address 127.17.109.195:38563)
0806 20:24:58.415923 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.415947 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.416228 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.416787 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.416794 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.416822 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:58.417121 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.417132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.418895 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.418900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.421350 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.421364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.423248 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.423258 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.423503 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.423708 (+ 205us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.423833 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.424038 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":120,"spinlock_wait_cycles":12032,"thread_start_us":1027,"threads_started":1}
I20260806 20:24:58.425297 19997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0c8d279994488f8dec8b7e0b146f37" candidate_uuid: "faa098785cd3413daed70a1a0fadf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" is_pre_election: true
I20260806 20:24:58.425628 19997 raft_consensus.cc:2454] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faa098785cd3413daed70a1a0fadf549 in term 0.
I20260806 20:24:58.425712 19850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0c8d279994488f8dec8b7e0b146f37" candidate_uuid: "faa098785cd3413daed70a1a0fadf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" is_pre_election: true
I20260806 20:24:58.426168 20071 leader_election.cc:302] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3b81068bc342a4b8362ca5aa0dcee9, faa098785cd3413daed70a1a0fadf549; no voters:
I20260806 20:24:58.426209 19850 raft_consensus.cc:2454] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faa098785cd3413daed70a1a0fadf549 in term 0.
I20260806 20:24:58.426684 20226 raft_consensus.cc:2781] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:24:58.426862 20226 raft_consensus.cc:482] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:24:58.426985 20226 raft_consensus.cc:3037] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.429378 20226 raft_consensus.cc:504] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.429996 20226 leader_election.cc:288] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [CANDIDATE]: Term 1 election: Requested vote from peers 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647), 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735)
I20260806 20:24:58.430600 19850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0c8d279994488f8dec8b7e0b146f37" candidate_uuid: "faa098785cd3413daed70a1a0fadf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243"
I20260806 20:24:58.430608 19997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed0c8d279994488f8dec8b7e0b146f37" candidate_uuid: "faa098785cd3413daed70a1a0fadf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9"
I20260806 20:24:58.430846 19850 raft_consensus.cc:3037] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.430886 19997 raft_consensus.cc:3037] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.433030 19850 raft_consensus.cc:2454] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faa098785cd3413daed70a1a0fadf549 in term 1.
I20260806 20:24:58.433126 19997 raft_consensus.cc:2454] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faa098785cd3413daed70a1a0fadf549 in term 1.
I20260806 20:24:58.433508 20072 leader_election.cc:302] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fc8c091aaf4b5cbe02340b0a2fd243, faa098785cd3413daed70a1a0fadf549; no voters:
I20260806 20:24:58.433828 20226 raft_consensus.cc:2781] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:58.434294 20226 raft_consensus.cc:686] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [term 1 LEADER]: Becoming Leader. State: Replica: faa098785cd3413daed70a1a0fadf549, State: Running, Role: LEADER
I20260806 20:24:58.434815 20226 consensus_queue.cc:237] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.438265 20242 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: TxnStatusTablet state changed. Reason: New leader faa098785cd3413daed70a1a0fadf549. Latest consensus state: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.438695 20242 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:24:58.439529 20244 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 20:24:58.439851 20244 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 20:24:58.440421 19686 catalog_manager.cc:5495] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 reported cstate change: term changed from 0 to 1, leader changed from <none> to faa098785cd3413daed70a1a0fadf549 (127.17.109.195). New cstate: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } health_report { overall_health: HEALTHY } } }
I20260806 20:24:58.491541 20236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.483490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:34375 (local address 127.0.0.1:55726)
0806 20:24:58.483815 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.483840 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.484021 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.484708 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.484717 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.484733 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:24:58.484959 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.484972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.486237 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.486243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.488069 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.488087 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.489264 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.489272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.489530 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.489616 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.490829 (+ 1213us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.490862 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.490982 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.491168 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260806 20:24:58.491640 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.483644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.17.109.195:34375)
0806 20:24:58.483834 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.483838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.483957 (+ 119us) server_negotiation.cc:407] Connection header received
0806 20:24:58.484316 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.484321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.484494 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.484638 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:58.485124 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.486082 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.488506 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.489034 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.489156 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.489868 (+ 712us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.490686 (+ 818us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.490848 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.491041 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.491203 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 20:24:58.501534 19997 raft_consensus.cc:1261] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Refusing update from remote peer faa098785cd3413daed70a1a0fadf549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:58.501833 19850 raft_consensus.cc:1261] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Refusing update from remote peer faa098785cd3413daed70a1a0fadf549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:58.502874 20226 consensus_queue.cc:1035] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:58.503404 20242 consensus_queue.cc:1035] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:58.516654 20225 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: TxnStatusTablet state changed. Reason: New leader faa098785cd3413daed70a1a0fadf549. Latest consensus state: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.517052 20225 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:58.519086 20247 mvcc.cc:205] Tried to move back new op lower bound from 7315652192250449920 to 7315652191993425920. Current Snapshot: MvccSnapshot[applied={T|T < 7315652192250449920}]
I20260806 20:24:58.520797 20224 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: TxnStatusTablet state changed. Reason: New leader faa098785cd3413daed70a1a0fadf549. Latest consensus state: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.521157 20224 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:58.524853 20226 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.525197 20226 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:24:58.528049 20242 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.530737 20242 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P faa098785cd3413daed70a1a0fadf549: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:24:58.531544 20224 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.531858 20224 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P 5a3b81068bc342a4b8362ca5aa0dcee9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:58.533324 20225 tablet_replica.cc:439] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } } }
I20260806 20:24:58.533676 20225 tablet_replica.cc:442] T ed0c8d279994488f8dec8b7e0b146f37 P 10fc8c091aaf4b5cbe02340b0a2fd243: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:24:58.543789 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.537161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41566 (local address 127.17.109.193:39647)
0806 20:24:58.537329 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.537334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.537993 (+ 659us) server_negotiation.cc:407] Connection header received
0806 20:24:58.538286 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.538290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.538384 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.538548 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:58.539038 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.539929 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.541442 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.541886 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.542036 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.542494 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.543040 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.543222 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.543313 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.543418 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 20:24:58.563109 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.556507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:34375 (local address 127.17.109.193:40815)
0806 20:24:58.557114 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.557131 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.557297 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.557681 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.557686 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.557698 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:24:58.558112 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.558132 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.559808 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.559812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.560865 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.560877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.561981 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.561985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.562110 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.562170 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.562632 (+ 462us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.562640 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.562749 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.562859 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":349,"threads_started":1}
I20260806 20:24:58.563419 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.556576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:40815 (local address 127.17.109.195:34375)
0806 20:24:58.556785 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.556794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.557226 (+ 432us) server_negotiation.cc:407] Connection header received
0806 20:24:58.557487 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.557491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.557558 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.557658 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:58.558334 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.559662 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.561012 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.561803 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.561959 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.562375 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.562523 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.562622 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.562802 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.562902 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 20:24:58.574910 19997 raft_consensus.cc:1261] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Refusing update from remote peer 10fc8c091aaf4b5cbe02340b0a2fd243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:58.575531 20145 raft_consensus.cc:1261] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Refusing update from remote peer 10fc8c091aaf4b5cbe02340b0a2fd243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:58.575645 20225 consensus_queue.cc:1035] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:58.576478 20225 consensus_queue.cc:1035] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:58.589978 20250 mvcc.cc:205] Tried to move back new op lower bound from 7315652192556552192 to 7315652190498926592. Current Snapshot: MvccSnapshot[applied={T|T < 7315652192556552192}]
I20260806 20:24:58.627673 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.620403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:50017 (local address 127.17.109.193:39647)
0806 20:24:58.620600 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.620609 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.621404 (+ 795us) server_negotiation.cc:407] Connection header received
0806 20:24:58.621643 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.621650 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.621744 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.621850 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:58.622674 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.623619 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.625488 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.626111 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.626274 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.626954 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.627059 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.627147 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.627296 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.627360 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 20:24:58.628012 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.620470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:39647 (local address 127.17.109.195:50017)
0806 20:24:58.621275 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.621297 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.621476 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.621871 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.621877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.621890 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:24:58.622442 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.622459 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.623776 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.623782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.625328 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.625341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.626461 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.626468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.626674 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.626749 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.627171 (+ 422us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.627185 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.627283 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.627445 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":367,"threads_started":1}
I20260806 20:24:58.670769 19685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:24:58.671093 19685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:24:58.675050 19685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: Sending DeleteTablet for 3 replicas of tablet c1cd4b0f99ef48eaabc55a45309e47e0
I20260806 20:24:58.676370 20123 tablet_service.cc:1514] Processing DeleteTablet for tablet c1cd4b0f99ef48eaabc55a45309e47e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:58 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:50646
I20260806 20:24:58.676362 19975 tablet_service.cc:1514] Processing DeleteTablet for tablet c1cd4b0f99ef48eaabc55a45309e47e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:58 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:49208
I20260806 20:24:58.677911 19828 tablet_service.cc:1514] Processing DeleteTablet for tablet c1cd4b0f99ef48eaabc55a45309e47e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:24:58 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:52380
I20260806 20:24:58.684506 19685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:24:58.687547 19685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:24:58.689162 20271 tablet_replica.cc:330] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: stopping tablet replica
I20260806 20:24:58.689729 20271 raft_consensus.cc:2229] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:24:58.690708 20271 raft_consensus.cc:2258] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:58.693472 20271 ts_tablet_manager.cc:1905] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:58.703083 19975 tablet_service.cc:1467] Processing CreateTablet for tablet e64a58c24f254f5d80cae0a1640c5e94 (DEFAULT_TABLE table=test-table [id=f2c1f3105ffb4bfa9c6d7f30c11d581b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:58.703483 19827 tablet_service.cc:1467] Processing CreateTablet for tablet e64a58c24f254f5d80cae0a1640c5e94 (DEFAULT_TABLE table=test-table [id=f2c1f3105ffb4bfa9c6d7f30c11d581b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:58.703661 19975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64a58c24f254f5d80cae0a1640c5e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:58.704035 19827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64a58c24f254f5d80cae0a1640c5e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:58.704115 20122 tablet_service.cc:1467] Processing CreateTablet for tablet e64a58c24f254f5d80cae0a1640c5e94 (DEFAULT_TABLE table=test-table [id=f2c1f3105ffb4bfa9c6d7f30c11d581b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:24:58.704628 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e64a58c24f254f5d80cae0a1640c5e94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:24:58.720891 20271 ts_tablet_manager.cc:1918] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:24:58.721149 20271 log.cc:1196] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/c1cd4b0f99ef48eaabc55a45309e47e0
I20260806 20:24:58.721853 20271 ts_tablet_manager.cc:1939] T c1cd4b0f99ef48eaabc55a45309e47e0 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting consensus metadata
I20260806 20:24:58.724119 20274 tablet_bootstrap.cc:492] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: Bootstrap starting.
I20260806 20:24:58.726339 20274 tablet_bootstrap.cc:654] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:58.727900 20276 tablet_bootstrap.cc:492] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: Bootstrap starting.
I20260806 20:24:58.729825 20273 tablet_replica.cc:330] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: stopping tablet replica
I20260806 20:24:58.730530 20273 raft_consensus.cc:2229] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:24:58.731475 20273 raft_consensus.cc:2258] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:58.734365 20276 tablet_bootstrap.cc:654] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:58.736074 20272 tablet_replica.cc:330] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: stopping tablet replica
I20260806 20:24:58.736528 20272 raft_consensus.cc:2229] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:24:58.736886 20272 raft_consensus.cc:2258] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:24:58.737112 20274 tablet_bootstrap.cc:492] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: No bootstrap required, opened a new log
I20260806 20:24:58.737286 20274 ts_tablet_manager.cc:1397] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260806 20:24:58.738282 20274 raft_consensus.cc:348] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.738585 20274 raft_consensus.cc:374] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.738690 20274 raft_consensus.cc:729] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fc8c091aaf4b5cbe02340b0a2fd243, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.739043 20274 consensus_queue.cc:260] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.739445 20272 ts_tablet_manager.cc:1905] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:58.739701 20274 ts_tablet_manager.cc:1428] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 20:24:58.740942 19670 catalog_manager.cc:4841] TS 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735): tablet c1cd4b0f99ef48eaabc55a45309e47e0 (table test-table [id=966b069e9f764d489c152062d7259883]) successfully deleted
I20260806 20:24:58.747377 20275 tablet_bootstrap.cc:492] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: Bootstrap starting.
I20260806 20:24:58.748379 20272 ts_tablet_manager.cc:1918] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:24:58.748124 20273 ts_tablet_manager.cc:1905] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:24:58.748711 20272 log.cc:1196] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/c1cd4b0f99ef48eaabc55a45309e47e0
I20260806 20:24:58.749869 20272 ts_tablet_manager.cc:1939] T c1cd4b0f99ef48eaabc55a45309e47e0 P faa098785cd3413daed70a1a0fadf549: Deleting consensus metadata
I20260806 20:24:58.750766 20275 tablet_bootstrap.cc:654] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: Neither blocks nor log segments found. Creating new log.
I20260806 20:24:58.752081 19673 catalog_manager.cc:4841] TS faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375): tablet c1cd4b0f99ef48eaabc55a45309e47e0 (table test-table [id=966b069e9f764d489c152062d7259883]) successfully deleted
I20260806 20:24:58.757213 20276 tablet_bootstrap.cc:492] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: No bootstrap required, opened a new log
I20260806 20:24:58.757412 20276 ts_tablet_manager.cc:1397] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.000s
I20260806 20:24:58.758220 20276 raft_consensus.cc:348] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.758508 20276 raft_consensus.cc:374] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.758651 20276 raft_consensus.cc:729] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa098785cd3413daed70a1a0fadf549, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.758956 20276 consensus_queue.cc:260] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.759656 20276 ts_tablet_manager.cc:1428] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 20:24:58.760166 20275 tablet_bootstrap.cc:492] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: No bootstrap required, opened a new log
I20260806 20:24:58.760342 20275 ts_tablet_manager.cc:1397] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260806 20:24:58.764285 20275 raft_consensus.cc:348] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.764549 20275 raft_consensus.cc:374] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:24:58.764693 20275 raft_consensus.cc:729] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3b81068bc342a4b8362ca5aa0dcee9, State: Initialized, Role: FOLLOWER
I20260806 20:24:58.768249 20273 ts_tablet_manager.cc:1918] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:24:58.768474 20273 log.cc:1196] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/c1cd4b0f99ef48eaabc55a45309e47e0
I20260806 20:24:58.769083 20273 ts_tablet_manager.cc:1939] T c1cd4b0f99ef48eaabc55a45309e47e0 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting consensus metadata
I20260806 20:24:58.770845 19671 catalog_manager.cc:4841] TS 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647): tablet c1cd4b0f99ef48eaabc55a45309e47e0 (table test-table [id=966b069e9f764d489c152062d7259883]) successfully deleted
I20260806 20:24:58.770809 20275 consensus_queue.cc:260] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.772286 20275 ts_tablet_manager.cc:1428] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260806 20:24:58.798413 20224 raft_consensus.cc:482] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:24:58.798715 20224 raft_consensus.cc:504] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.800745 20224 leader_election.cc:288] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647), faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375)
I20260806 20:24:58.817898 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.803114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:41643 (local address 127.17.109.193:39647)
0806 20:24:58.803298 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.803304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.809575 (+ 6271us) server_negotiation.cc:407] Connection header received
0806 20:24:58.809880 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.809887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.809978 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.810051 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:58.810955 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.812192 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.814863 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.816466 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.816651 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.816744 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.817369 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.817525 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:24:58.819335 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.802647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:39647 (local address 127.17.109.194:41643)
0806 20:24:58.809458 (+ 6811us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.809479 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.809689 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.810280 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.810285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.810359 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:58.810752 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.810765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.812426 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.812431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.814662 (+ 2231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.814679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.816937 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.816946 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.817150 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.817187 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.818698 (+ 1511us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.818867 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6676,"mutex_wait_us":27,"thread_start_us":6466,"threads_started":1}
I20260806 20:24:58.819391 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.801909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:37277 (local address 127.17.109.195:34375)
0806 20:24:58.802123 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.802131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.807097 (+ 4966us) server_negotiation.cc:407] Connection header received
0806 20:24:58.807505 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.807516 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.807655 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.807744 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:24:58.809448 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.810835 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.814000 (+ 3165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.815760 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.815909 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.816008 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.818899 (+ 2891us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.819051 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 20:24:58.820263 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.801355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:34375 (local address 127.17.109.194:37277)
0806 20:24:58.806941 (+ 5586us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.806977 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.807266 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.808032 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.808037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.808078 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:24:58.808663 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.808686 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.811211 (+ 2525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.811216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.813718 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.813736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.818478 (+ 4742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.818484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.818686 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.818723 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.819689 (+ 966us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.819853 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5419,"mutex_wait_us":31,"thread_start_us":5266,"threads_started":1}
I20260806 20:24:58.820430 19850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64a58c24f254f5d80cae0a1640c5e94" candidate_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" is_pre_election: true
I20260806 20:24:58.820735 19850 raft_consensus.cc:2454] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3b81068bc342a4b8362ca5aa0dcee9 in term 0.
I20260806 20:24:58.821111 20145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64a58c24f254f5d80cae0a1640c5e94" candidate_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa098785cd3413daed70a1a0fadf549" is_pre_election: true
I20260806 20:24:58.821406 20145 raft_consensus.cc:2454] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3b81068bc342a4b8362ca5aa0dcee9 in term 0.
I20260806 20:24:58.821372 19924 leader_election.cc:302] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fc8c091aaf4b5cbe02340b0a2fd243, 5a3b81068bc342a4b8362ca5aa0dcee9; no voters:
I20260806 20:24:58.821883 20224 raft_consensus.cc:2781] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:24:58.822050 20224 raft_consensus.cc:482] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:24:58.822182 20224 raft_consensus.cc:3037] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.824657 20224 raft_consensus.cc:504] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.825892 19850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64a58c24f254f5d80cae0a1640c5e94" candidate_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243"
I20260806 20:24:58.825892 20145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e64a58c24f254f5d80cae0a1640c5e94" candidate_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa098785cd3413daed70a1a0fadf549"
I20260806 20:24:58.826145 20145 raft_consensus.cc:3037] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.826140 19850 raft_consensus.cc:3037] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:24:58.829530 20224 leader_election.cc:288] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [CANDIDATE]: Term 1 election: Requested vote from peers 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647), faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375)
I20260806 20:24:58.829571 19850 raft_consensus.cc:2454] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3b81068bc342a4b8362ca5aa0dcee9 in term 1.
I20260806 20:24:58.829845 20145 raft_consensus.cc:2454] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3b81068bc342a4b8362ca5aa0dcee9 in term 1.
I20260806 20:24:58.830229 19924 leader_election.cc:302] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fc8c091aaf4b5cbe02340b0a2fd243, 5a3b81068bc342a4b8362ca5aa0dcee9; no voters:
I20260806 20:24:58.830603 20224 raft_consensus.cc:2781] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:24:58.831436 20224 raft_consensus.cc:686] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 LEADER]: Becoming Leader. State: Replica: 5a3b81068bc342a4b8362ca5aa0dcee9, State: Running, Role: LEADER
I20260806 20:24:58.831939 20224 consensus_queue.cc:237] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:24:58.836167 19685 catalog_manager.cc:5495] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } health_report { overall_health: UNKNOWN } } }
I20260806 20:24:58.866358 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.857654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.17.109.194:44735)
0806 20:24:58.857831 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.857838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.857927 (+ 89us) server_negotiation.cc:407] Connection header received
0806 20:24:58.858174 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.858179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.858282 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.858458 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:58.858986 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.860038 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.862204 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.862957 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.863113 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.863830 (+ 717us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.864789 (+ 959us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.864950 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.865762 (+ 812us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.865926 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 20:24:58.907359 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.895318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:54451 (local address 127.17.109.195:34375)
0806 20:24:58.895543 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.895574 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.901871 (+ 6297us) server_negotiation.cc:407] Connection header received
0806 20:24:58.902123 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.902130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.902230 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.902352 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:58.902937 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.903880 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.905425 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.905878 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.906037 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.906540 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.906665 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.906775 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.906927 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.907013 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 20:24:58.907480 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.895331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:34375 (local address 127.17.109.194:54451)
0806 20:24:58.901740 (+ 6409us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.901760 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.901962 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.902370 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.902375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.902387 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:24:58.902771 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.902783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.904046 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.904055 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.905260 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.905275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.906089 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.906098 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.906268 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.906328 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.906777 (+ 449us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.906793 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.906903 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.907048 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6241,"thread_start_us":5953,"threads_started":1}
I20260806 20:24:58.919880 20145 raft_consensus.cc:1261] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Refusing update from remote peer 5a3b81068bc342a4b8362ca5aa0dcee9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:58.920004 19850 raft_consensus.cc:1261] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Refusing update from remote peer 5a3b81068bc342a4b8362ca5aa0dcee9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:24:58.920801 20282 consensus_queue.cc:1035] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:58.921319 20224 consensus_queue.cc:1035] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:24:58.939229 20248 mvcc.cc:205] Tried to move back new op lower bound from 7315652193968463872 to 7315652193620692992. Current Snapshot: MvccSnapshot[applied={T|T < 7315652193968463872}]
I20260806 20:24:58.989322 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.982884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44735 (local address 127.17.109.195:34909)
0806 20:24:58.983095 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 20:24:58.983117 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:24:58.983265 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:24:58.983654 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:24:58.983659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:24:58.983676 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:24:58.983890 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.983903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.985324 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.985332 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.987044 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:24:58.987060 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.987944 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.987951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:24:58.988167 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.988217 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.988725 (+ 508us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.988738 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 20:24:58.988826 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 20:24:58.988980 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 20:24:58.990087 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:24:58.983133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:34909 (local address 127.17.109.194:44735)
0806 20:24:58.983327 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 20:24:58.983336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:24:58.983354 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:24:58.983418 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:24:58.983422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:24:58.983497 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:24:58.983625 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:24:58.984074 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.985152 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.987190 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:24:58.987772 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:24:58.987923 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:24:58.988415 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:24:58.988573 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:24:58.988662 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 20:24:58.989720 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0806 20:24:58.989809 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 20:24:58.992128 20248 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 20:25:00.618389 19685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:00.618633 19685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:00.622959 19685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: Sending DeleteTablet for 3 replicas of tablet e64a58c24f254f5d80cae0a1640c5e94
I20260806 20:25:00.623819 19828 tablet_service.cc:1514] Processing DeleteTablet for tablet e64a58c24f254f5d80cae0a1640c5e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:00 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:52380
I20260806 20:25:00.623991 19975 tablet_service.cc:1514] Processing DeleteTablet for tablet e64a58c24f254f5d80cae0a1640c5e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:00 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:49208
I20260806 20:25:00.624013 20123 tablet_service.cc:1514] Processing DeleteTablet for tablet e64a58c24f254f5d80cae0a1640c5e94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:00 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:50646
I20260806 20:25:00.635036 20314 tablet_replica.cc:330] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: stopping tablet replica
I20260806 20:25:00.635514 20314 raft_consensus.cc:2229] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:00.636363 20314 raft_consensus.cc:2258] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:00.637094 20313 tablet_replica.cc:330] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: stopping tablet replica
I20260806 20:25:00.637365 20313 raft_consensus.cc:2229] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:00.637678 20314 ts_tablet_manager.cc:1905] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:00.637677 20313 raft_consensus.cc:2258] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:00.638797 20313 ts_tablet_manager.cc:1905] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:00.639689 20315 tablet_replica.cc:330] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: stopping tablet replica
I20260806 20:25:00.640269 20315 raft_consensus.cc:2229] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:00.640534 20315 raft_consensus.cc:2258] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:00.641543 20315 ts_tablet_manager.cc:1905] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:00.644652 20314 ts_tablet_manager.cc:1918] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 20:25:00.644809 20314 log.cc:1196] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/e64a58c24f254f5d80cae0a1640c5e94
I20260806 20:25:00.645295 20314 ts_tablet_manager.cc:1939] T e64a58c24f254f5d80cae0a1640c5e94 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting consensus metadata
I20260806 20:25:00.645928 20313 ts_tablet_manager.cc:1918] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 20:25:00.646082 20313 log.cc:1196] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/e64a58c24f254f5d80cae0a1640c5e94
I20260806 20:25:00.646579 20313 ts_tablet_manager.cc:1939] T e64a58c24f254f5d80cae0a1640c5e94 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting consensus metadata
I20260806 20:25:00.646622 19670 catalog_manager.cc:4841] TS 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735): tablet e64a58c24f254f5d80cae0a1640c5e94 (table test-table [id=f2c1f3105ffb4bfa9c6d7f30c11d581b]) successfully deleted
I20260806 20:25:00.647718 19671 catalog_manager.cc:4841] TS 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647): tablet e64a58c24f254f5d80cae0a1640c5e94 (table test-table [id=f2c1f3105ffb4bfa9c6d7f30c11d581b]) successfully deleted
I20260806 20:25:00.649158 20315 ts_tablet_manager.cc:1918] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 20:25:00.649297 20315 log.cc:1196] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/e64a58c24f254f5d80cae0a1640c5e94
I20260806 20:25:00.649909 20315 ts_tablet_manager.cc:1939] T e64a58c24f254f5d80cae0a1640c5e94 P faa098785cd3413daed70a1a0fadf549: Deleting consensus metadata
I20260806 20:25:00.651085 19673 catalog_manager.cc:4841] TS faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375): tablet e64a58c24f254f5d80cae0a1640c5e94 (table test-table [id=f2c1f3105ffb4bfa9c6d7f30c11d581b]) successfully deleted
Aug 06 20:25:00 dist-test-slave-6k22 krb5kdc[19640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047900, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 20:25:00 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047900, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:00.694641 20322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:00.678416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37448 (local address 127.17.109.254:46533)
0806 20:25:00.679164 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:00.679170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:00.679244 (+ 74us) server_negotiation.cc:407] Connection header received
0806 20:25:00.679428 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:00.679433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:00.679526 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:00.679758 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:00.681188 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:00.682068 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:00.683467 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:00.684071 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:00.684192 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:00.688556 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:00.688561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:00.688565 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:00.688602 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:00.691711 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:00.692441 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:00.692447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:00.692451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:00.692596 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:00.693024 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:00.693030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:00.693033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:00.693330 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:00.693625 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:00.693949 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 20:25:00.694206 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":399,"threads_started":1}
I20260806 20:25:00.698994 19685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:25:00.699661 19685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:00.707046 19828 tablet_service.cc:1467] Processing CreateTablet for tablet 6a44adfd9b124126a3b66e6496b65c40 (DEFAULT_TABLE table=test-table [id=33027d5c108a4a168b9d9ee30aedcb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:00.707158 19975 tablet_service.cc:1467] Processing CreateTablet for tablet 6a44adfd9b124126a3b66e6496b65c40 (DEFAULT_TABLE table=test-table [id=33027d5c108a4a168b9d9ee30aedcb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:00.707439 20123 tablet_service.cc:1467] Processing CreateTablet for tablet 6a44adfd9b124126a3b66e6496b65c40 (DEFAULT_TABLE table=test-table [id=33027d5c108a4a168b9d9ee30aedcb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:00.707604 19828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a44adfd9b124126a3b66e6496b65c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:00.707726 19975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a44adfd9b124126a3b66e6496b65c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:00.707937 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a44adfd9b124126a3b66e6496b65c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:00.713655 20324 tablet_bootstrap.cc:492] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: Bootstrap starting.
I20260806 20:25:00.715792 20324 tablet_bootstrap.cc:654] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:00.717760 20326 tablet_bootstrap.cc:492] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: Bootstrap starting.
I20260806 20:25:00.718834 20324 tablet_bootstrap.cc:492] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: No bootstrap required, opened a new log
I20260806 20:25:00.719014 20324 ts_tablet_manager.cc:1397] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260806 20:25:00.719769 20324 raft_consensus.cc:348] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.720012 20324 raft_consensus.cc:374] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:00.720137 20324 raft_consensus.cc:729] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3b81068bc342a4b8362ca5aa0dcee9, State: Initialized, Role: FOLLOWER
I20260806 20:25:00.720436 20324 consensus_queue.cc:260] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.721012 20324 ts_tablet_manager.cc:1428] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 20:25:00.722126 20325 tablet_bootstrap.cc:492] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: Bootstrap starting.
I20260806 20:25:00.722667 20326 tablet_bootstrap.cc:654] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:00.724128 20325 tablet_bootstrap.cc:654] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:00.727181 20325 tablet_bootstrap.cc:492] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: No bootstrap required, opened a new log
I20260806 20:25:00.727371 20325 ts_tablet_manager.cc:1397] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 20:25:00.728401 20325 raft_consensus.cc:348] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.728708 20325 raft_consensus.cc:374] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:00.728823 20325 raft_consensus.cc:729] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fc8c091aaf4b5cbe02340b0a2fd243, State: Initialized, Role: FOLLOWER
I20260806 20:25:00.729043 20326 tablet_bootstrap.cc:492] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: No bootstrap required, opened a new log
I20260806 20:25:00.729192 20326 ts_tablet_manager.cc:1397] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260806 20:25:00.729096 20325 consensus_queue.cc:260] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.729885 20326 raft_consensus.cc:348] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.730142 20325 ts_tablet_manager.cc:1428] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 20:25:00.730159 20326 raft_consensus.cc:374] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:00.730237 20326 raft_consensus.cc:729] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa098785cd3413daed70a1a0fadf549, State: Initialized, Role: FOLLOWER
I20260806 20:25:00.730466 20326 consensus_queue.cc:260] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.731061 20326 ts_tablet_manager.cc:1428] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 20:25:00.756522 20331 raft_consensus.cc:482] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:00.756822 20331 raft_consensus.cc:504] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.757638 20331 leader_election.cc:288] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735), faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375)
I20260806 20:25:00.758276 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a44adfd9b124126a3b66e6496b65c40" candidate_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa098785cd3413daed70a1a0fadf549" is_pre_election: true
I20260806 20:25:00.758286 19996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a44adfd9b124126a3b66e6496b65c40" candidate_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" is_pre_election: true
I20260806 20:25:00.758577 20143 raft_consensus.cc:2454] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10fc8c091aaf4b5cbe02340b0a2fd243 in term 0.
I20260806 20:25:00.758581 19996 raft_consensus.cc:2454] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10fc8c091aaf4b5cbe02340b0a2fd243 in term 0.
I20260806 20:25:00.759119 19776 leader_election.cc:302] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fc8c091aaf4b5cbe02340b0a2fd243, 5a3b81068bc342a4b8362ca5aa0dcee9; no voters:
I20260806 20:25:00.759405 20331 raft_consensus.cc:2781] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:00.759534 20331 raft_consensus.cc:482] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:00.759675 20331 raft_consensus.cc:3037] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:00.762068 20331 raft_consensus.cc:504] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.762677 20331 leader_election.cc:288] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [CANDIDATE]: Term 1 election: Requested vote from peers 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735), faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375)
I20260806 20:25:00.763320 19996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a44adfd9b124126a3b66e6496b65c40" candidate_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9"
I20260806 20:25:00.763391 20143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a44adfd9b124126a3b66e6496b65c40" candidate_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faa098785cd3413daed70a1a0fadf549"
I20260806 20:25:00.763546 19996 raft_consensus.cc:3037] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:00.763775 20143 raft_consensus.cc:3037] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:00.766129 19996 raft_consensus.cc:2454] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10fc8c091aaf4b5cbe02340b0a2fd243 in term 1.
I20260806 20:25:00.766532 19776 leader_election.cc:302] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10fc8c091aaf4b5cbe02340b0a2fd243, 5a3b81068bc342a4b8362ca5aa0dcee9; no voters:
I20260806 20:25:00.766590 20143 raft_consensus.cc:2454] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10fc8c091aaf4b5cbe02340b0a2fd243 in term 1.
I20260806 20:25:00.766964 20331 raft_consensus.cc:2781] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:00.767503 20331 raft_consensus.cc:686] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 LEADER]: Becoming Leader. State: Replica: 10fc8c091aaf4b5cbe02340b0a2fd243, State: Running, Role: LEADER
I20260806 20:25:00.767867 20331 consensus_queue.cc:237] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.770494 19686 catalog_manager.cc:5495] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193). New cstate: current_term: 1 leader_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } health_report { overall_health: UNKNOWN } } }
I20260806 20:25:00.800679 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:00.794122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41580 (local address 127.17.109.193:39647)
0806 20:25:00.794919 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:00.794927 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:00.794949 (+ 22us) server_negotiation.cc:407] Connection header received
0806 20:25:00.795017 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:00.795023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:00.795107 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:00.795188 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:00.795672 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:00.796526 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:00.798453 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:00.799066 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:00.799170 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:00.799791 (+ 621us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:00.799912 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:00.799991 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:00.800192 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 20:25:00.800259 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":494,"threads_started":1}
I20260806 20:25:00.810736 20143 raft_consensus.cc:1261] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Refusing update from remote peer 10fc8c091aaf4b5cbe02340b0a2fd243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:00.810722 19996 raft_consensus.cc:1261] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Refusing update from remote peer 10fc8c091aaf4b5cbe02340b0a2fd243: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:00.811326 20332 consensus_queue.cc:1035] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:00.811822 20331 consensus_queue.cc:1035] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:00.859095 19686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:00.859386 19686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:00.862360 19686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: Sending DeleteTablet for 3 replicas of tablet 6a44adfd9b124126a3b66e6496b65c40
I20260806 20:25:00.863233 20123 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a44adfd9b124126a3b66e6496b65c40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:00 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:50646
I20260806 20:25:00.863206 19828 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a44adfd9b124126a3b66e6496b65c40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:00 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:52380
I20260806 20:25:00.863266 19975 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a44adfd9b124126a3b66e6496b65c40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:00 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:49208
I20260806 20:25:00.863533 20314 tablet_replica.cc:330] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: stopping tablet replica
I20260806 20:25:00.863544 20315 tablet_replica.cc:330] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: stopping tablet replica
I20260806 20:25:00.863770 20314 raft_consensus.cc:2229] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:00.863775 20315 raft_consensus.cc:2229] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:00.863978 20315 raft_consensus.cc:2258] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:00.863978 20314 raft_consensus.cc:2258] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:00.864257 20313 tablet_replica.cc:330] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: stopping tablet replica
I20260806 20:25:00.864465 20313 raft_consensus.cc:2229] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:00.864781 20313 raft_consensus.cc:2258] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:00.864997 20315 ts_tablet_manager.cc:1905] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:00.866392 20313 ts_tablet_manager.cc:1905] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:00.872012 20315 ts_tablet_manager.cc:1918] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:00.871902 19686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 20:25:00.872179 20315 log.cc:1196] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/6a44adfd9b124126a3b66e6496b65c40
I20260806 20:25:00.872648 20315 ts_tablet_manager.cc:1939] T 6a44adfd9b124126a3b66e6496b65c40 P faa098785cd3413daed70a1a0fadf549: Deleting consensus metadata
W20260806 20:25:00.872646 19686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:00.873102 20314 ts_tablet_manager.cc:1905] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:00.873859 19673 catalog_manager.cc:4841] TS faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375): tablet 6a44adfd9b124126a3b66e6496b65c40 (table test-table [id=33027d5c108a4a168b9d9ee30aedcb0c]) successfully deleted
I20260806 20:25:00.875391 20313 ts_tablet_manager.cc:1918] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:00.877664 20313 log.cc:1196] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/6a44adfd9b124126a3b66e6496b65c40
I20260806 20:25:00.879608 19975 tablet_service.cc:1467] Processing CreateTablet for tablet 5793e82d29574bdab6c42515c8114775 (DEFAULT_TABLE table=test-table [id=a73a89c67a114251ab3b1a37d225b684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:00.879748 20123 tablet_service.cc:1467] Processing CreateTablet for tablet 5793e82d29574bdab6c42515c8114775 (DEFAULT_TABLE table=test-table [id=a73a89c67a114251ab3b1a37d225b684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:00.879590 19828 tablet_service.cc:1467] Processing CreateTablet for tablet 5793e82d29574bdab6c42515c8114775 (DEFAULT_TABLE table=test-table [id=a73a89c67a114251ab3b1a37d225b684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:00.880057 19975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5793e82d29574bdab6c42515c8114775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:00.880184 20123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5793e82d29574bdab6c42515c8114775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:00.880327 19828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5793e82d29574bdab6c42515c8114775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:00.880483 20314 ts_tablet_manager.cc:1918] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:00.880685 20314 log.cc:1196] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/6a44adfd9b124126a3b66e6496b65c40
I20260806 20:25:00.881242 20314 ts_tablet_manager.cc:1939] T 6a44adfd9b124126a3b66e6496b65c40 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting consensus metadata
I20260806 20:25:00.882702 19670 catalog_manager.cc:4841] TS 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735): tablet 6a44adfd9b124126a3b66e6496b65c40 (table test-table [id=33027d5c108a4a168b9d9ee30aedcb0c]) successfully deleted
I20260806 20:25:00.883616 20313 ts_tablet_manager.cc:1939] T 6a44adfd9b124126a3b66e6496b65c40 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting consensus metadata
I20260806 20:25:00.884804 19671 catalog_manager.cc:4841] TS 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647): tablet 6a44adfd9b124126a3b66e6496b65c40 (table test-table [id=33027d5c108a4a168b9d9ee30aedcb0c]) successfully deleted
I20260806 20:25:00.886439 20326 tablet_bootstrap.cc:492] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: Bootstrap starting.
I20260806 20:25:00.886924 20325 tablet_bootstrap.cc:492] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: Bootstrap starting.
I20260806 20:25:00.887176 20324 tablet_bootstrap.cc:492] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: Bootstrap starting.
I20260806 20:25:00.888353 20326 tablet_bootstrap.cc:654] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:00.888463 20325 tablet_bootstrap.cc:654] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:00.889048 20324 tablet_bootstrap.cc:654] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:00.890643 20325 tablet_bootstrap.cc:492] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: No bootstrap required, opened a new log
I20260806 20:25:00.890815 20325 ts_tablet_manager.cc:1397] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 20:25:00.891481 20325 raft_consensus.cc:348] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.891798 20325 raft_consensus.cc:374] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:00.891886 20325 raft_consensus.cc:729] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10fc8c091aaf4b5cbe02340b0a2fd243, State: Initialized, Role: FOLLOWER
I20260806 20:25:00.892153 20324 tablet_bootstrap.cc:492] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: No bootstrap required, opened a new log
I20260806 20:25:00.892124 20325 consensus_queue.cc:260] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.892431 20324 ts_tablet_manager.cc:1397] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 20:25:00.892650 20325 ts_tablet_manager.cc:1428] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 20:25:00.894029 20324 raft_consensus.cc:348] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.894284 20326 tablet_bootstrap.cc:492] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: No bootstrap required, opened a new log
I20260806 20:25:00.894330 20324 raft_consensus.cc:374] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:00.894438 20324 raft_consensus.cc:729] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3b81068bc342a4b8362ca5aa0dcee9, State: Initialized, Role: FOLLOWER
I20260806 20:25:00.894461 20326 ts_tablet_manager.cc:1397] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260806 20:25:00.894868 20324 consensus_queue.cc:260] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.895229 20326 raft_consensus.cc:348] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.895524 20326 raft_consensus.cc:374] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:00.895682 20326 raft_consensus.cc:729] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa098785cd3413daed70a1a0fadf549, State: Initialized, Role: FOLLOWER
I20260806 20:25:00.895955 20326 consensus_queue.cc:260] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:00.896505 20326 ts_tablet_manager.cc:1428] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 20:25:00.896214 20324 ts_tablet_manager.cc:1428] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 20:25:01.270138 20290 raft_consensus.cc:482] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:01.270435 20290 raft_consensus.cc:504] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:01.271171 20290 leader_election.cc:288] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647), 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735)
I20260806 20:25:01.271984 19996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5793e82d29574bdab6c42515c8114775" candidate_uuid: "faa098785cd3413daed70a1a0fadf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" is_pre_election: true
I20260806 20:25:01.271984 19849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5793e82d29574bdab6c42515c8114775" candidate_uuid: "faa098785cd3413daed70a1a0fadf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" is_pre_election: true
I20260806 20:25:01.272282 19849 raft_consensus.cc:2454] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faa098785cd3413daed70a1a0fadf549 in term 0.
I20260806 20:25:01.272290 19996 raft_consensus.cc:2454] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faa098785cd3413daed70a1a0fadf549 in term 0.
I20260806 20:25:01.272878 20072 leader_election.cc:302] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10fc8c091aaf4b5cbe02340b0a2fd243, 5a3b81068bc342a4b8362ca5aa0dcee9, faa098785cd3413daed70a1a0fadf549; no voters:
I20260806 20:25:01.273228 20290 raft_consensus.cc:2781] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:01.273371 20290 raft_consensus.cc:482] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:01.273444 20290 raft_consensus.cc:3037] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:01.275763 20290 raft_consensus.cc:504] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:01.276400 20290 leader_election.cc:288] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [CANDIDATE]: Term 1 election: Requested vote from peers 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647), 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735)
I20260806 20:25:01.276980 19849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5793e82d29574bdab6c42515c8114775" candidate_uuid: "faa098785cd3413daed70a1a0fadf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243"
I20260806 20:25:01.277004 19996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5793e82d29574bdab6c42515c8114775" candidate_uuid: "faa098785cd3413daed70a1a0fadf549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9"
I20260806 20:25:01.277220 19849 raft_consensus.cc:3037] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:01.277248 19996 raft_consensus.cc:3037] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:01.279417 19996 raft_consensus.cc:2454] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faa098785cd3413daed70a1a0fadf549 in term 1.
I20260806 20:25:01.279762 19849 raft_consensus.cc:2454] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faa098785cd3413daed70a1a0fadf549 in term 1.
I20260806 20:25:01.279887 20071 leader_election.cc:302] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3b81068bc342a4b8362ca5aa0dcee9, faa098785cd3413daed70a1a0fadf549; no voters:
I20260806 20:25:01.280190 20290 raft_consensus.cc:2781] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:01.280370 20290 raft_consensus.cc:686] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 1 LEADER]: Becoming Leader. State: Replica: faa098785cd3413daed70a1a0fadf549, State: Running, Role: LEADER
I20260806 20:25:01.280665 20290 consensus_queue.cc:237] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } }
I20260806 20:25:01.283541 19686 catalog_manager.cc:5495] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 reported cstate change: term changed from 0 to 1, leader changed from <none> to faa098785cd3413daed70a1a0fadf549 (127.17.109.195). New cstate: current_term: 1 leader_uuid: "faa098785cd3413daed70a1a0fadf549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faa098785cd3413daed70a1a0fadf549" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 34375 } health_report { overall_health: HEALTHY } } }
I20260806 20:25:01.366956 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:01.360847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.17.109.195:34375)
0806 20:25:01.361356 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:01.361363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:01.361530 (+ 167us) server_negotiation.cc:407] Connection header received
0806 20:25:01.361742 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:01.361745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:01.361811 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:01.361899 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:01.362388 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:01.363263 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:01.364994 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:01.365399 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:01.365501 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:01.366084 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:01.366189 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:01.366266 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:01.366459 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 20:25:01.366526 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":254,"threads_started":1}
I20260806 20:25:01.381078 20350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:01.375855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:56985 (local address 127.17.109.195:34375)
0806 20:25:01.376083 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:01.376091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:01.376454 (+ 363us) server_negotiation.cc:407] Connection header received
0806 20:25:01.376644 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:01.376649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:01.376715 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:01.376813 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:01.377267 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:01.378010 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:01.379329 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:01.379794 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:01.379932 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:01.380322 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:01.380422 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:01.380520 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:01.380657 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 20:25:01.380736 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":49}
I20260806 20:25:01.381131 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:01.375739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:34375 (local address 127.17.109.195:56985)
0806 20:25:01.376359 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:01.376374 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:01.376528 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:01.376854 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:01.376858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:01.376869 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:25:01.377105 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:01.377113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:01.378144 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:01.378151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:01.379164 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:01.379175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:01.379974 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:01.379982 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:01.380106 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:01.380137 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:25:01.380520 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:25:01.380532 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 20:25:01.380631 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 20:25:01.380753 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":343,"threads_started":1}
I20260806 20:25:01.388537 19849 raft_consensus.cc:1261] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Refusing update from remote peer faa098785cd3413daed70a1a0fadf549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:01.388571 19996 raft_consensus.cc:1261] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Refusing update from remote peer faa098785cd3413daed70a1a0fadf549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:01.389068 20346 consensus_queue.cc:1035] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10fc8c091aaf4b5cbe02340b0a2fd243" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:01.389408 20294 consensus_queue.cc:1035] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3b81068bc342a4b8362ca5aa0dcee9" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 20:25:01.410146 20292 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 20:25:03.017352 19685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:03.017580 19685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:03.022012 19685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e674a5d0b4f5492484cfe3d1c0c67a97: Sending DeleteTablet for 3 replicas of tablet 5793e82d29574bdab6c42515c8114775
I20260806 20:25:03.022934 19828 tablet_service.cc:1514] Processing DeleteTablet for tablet 5793e82d29574bdab6c42515c8114775 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:03 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:52380
I20260806 20:25:03.022975 20122 tablet_service.cc:1514] Processing DeleteTablet for tablet 5793e82d29574bdab6c42515c8114775 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:03 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:50646
I20260806 20:25:03.023011 19975 tablet_service.cc:1514] Processing DeleteTablet for tablet 5793e82d29574bdab6c42515c8114775 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:03 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:49208
I20260806 20:25:03.024168 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19759
I20260806 20:25:03.035493 20384 tablet_replica.cc:330] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: stopping tablet replica
I20260806 20:25:03.035969 20384 raft_consensus.cc:2229] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:03.036442 20384 raft_consensus.cc:2258] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:03.037174 20382 tablet_replica.cc:330] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: stopping tablet replica
I20260806 20:25:03.037443 20382 raft_consensus.cc:2229] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:03.037783 20382 raft_consensus.cc:2258] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:03.037950 20384 ts_tablet_manager.cc:1905] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:03.038908 20382 ts_tablet_manager.cc:1905] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:03.041473 20383 tablet_replica.cc:330] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: stopping tablet replica
I20260806 20:25:03.041771 20383 raft_consensus.cc:2229] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:03.042971 20383 raft_consensus.cc:2258] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:03.044253 20383 ts_tablet_manager.cc:1905] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:03.045372 20382 ts_tablet_manager.cc:1918] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 20:25:03.045526 20382 log.cc:1196] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/5793e82d29574bdab6c42515c8114775
I20260806 20:25:03.045941 20382 ts_tablet_manager.cc:1939] T 5793e82d29574bdab6c42515c8114775 P 10fc8c091aaf4b5cbe02340b0a2fd243: Deleting consensus metadata
I20260806 20:25:03.046948 19671 catalog_manager.cc:4841] TS 10fc8c091aaf4b5cbe02340b0a2fd243 (127.17.109.193:39647): tablet 5793e82d29574bdab6c42515c8114775 (table test-table [id=a73a89c67a114251ab3b1a37d225b684]) successfully deleted
I20260806 20:25:03.047538 20384 ts_tablet_manager.cc:1918] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 20:25:03.047734 20384 log.cc:1196] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/5793e82d29574bdab6c42515c8114775
I20260806 20:25:03.048338 20384 ts_tablet_manager.cc:1939] T 5793e82d29574bdab6c42515c8114775 P 5a3b81068bc342a4b8362ca5aa0dcee9: Deleting consensus metadata
I20260806 20:25:03.049867 19670 catalog_manager.cc:4841] TS 5a3b81068bc342a4b8362ca5aa0dcee9 (127.17.109.194:44735): tablet 5793e82d29574bdab6c42515c8114775 (table test-table [id=a73a89c67a114251ab3b1a37d225b684]) successfully deleted
I20260806 20:25:03.051429 20383 ts_tablet_manager.cc:1918] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 20:25:03.051643 20383 log.cc:1196] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/5793e82d29574bdab6c42515c8114775
I20260806 20:25:03.052209 20383 ts_tablet_manager.cc:1939] T 5793e82d29574bdab6c42515c8114775 P faa098785cd3413daed70a1a0fadf549: Deleting consensus metadata
Aug 06 20:25:03 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047900, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:25:03.053471 19673 catalog_manager.cc:4841] TS faa098785cd3413daed70a1a0fadf549 (127.17.109.195:34375): tablet 5793e82d29574bdab6c42515c8114775 (table test-table [id=a73a89c67a114251ab3b1a37d225b684]) successfully deleted
I20260806 20:25:03.059674 20386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:03.025776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41582 (local address 127.17.109.193:39647)
0806 20:25:03.035173 (+ 9397us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:03.035181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:03.035211 (+ 30us) server_negotiation.cc:407] Connection header received
0806 20:25:03.035291 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:03.035299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:03.035409 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:03.035544 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:03.037775 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:03.038961 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:03.040783 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:03.041441 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:03.041556 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:03.053457 (+ 11901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:03.053475 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:03.053491 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:03.053530 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:03.056679 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:03.057435 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:03.057443 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:03.057445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:03.057579 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:03.057988 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:03.057992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:03.057994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:03.058377 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:03.058683 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:03.059005 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 20:25:03.059260 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9209,"thread_start_us":8970,"threads_started":1}
W20260806 20:25:03.060439 19898 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41582
W20260806 20:25:03.061254 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19759: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:25:03.061466 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 19759
I20260806 20:25:03.089192 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19907
Aug 06 20:25:03 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047900, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:25:03.106678 20387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:03.090095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50386 (local address 127.17.109.194:44735)
0806 20:25:03.090721 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:03.090726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:03.090743 (+ 17us) server_negotiation.cc:407] Connection header received
0806 20:25:03.090797 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:03.090800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:03.090858 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:03.090951 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:03.092514 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:03.093316 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:03.094862 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:03.095396 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:03.095519 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:03.100360 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:03.100375 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:03.100392 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:03.100432 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:03.103510 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:03.104345 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:03.104353 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:03.104356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:03.104569 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:03.104980 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:03.104984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:03.104986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:03.105469 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:03.105721 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:03.106072 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 20:25:03.106270 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":330,"threads_started":1}
W20260806 20:25:03.107462 20043 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50386
W20260806 20:25:03.108314 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19907: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:25:03.108538 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 19907
I20260806 20:25:03.134354 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20053
Aug 06 20:25:03 dist-test-slave-6k22 krb5kdc[19640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047900, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:25:03.150242 20388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:03.135189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55732 (local address 127.17.109.195:34375)
0806 20:25:03.135787 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:03.135793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:03.135810 (+ 17us) server_negotiation.cc:407] Connection header received
0806 20:25:03.135873 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:03.135876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:03.135932 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:03.136050 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:03.137371 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:03.138135 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:03.139373 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:03.139932 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:03.140070 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:03.144480 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:03.144494 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:03.144510 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:03.144548 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:03.147472 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:03.148138 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:03.148159 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:03.148161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:03.148290 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:03.148680 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:03.148687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:03.148690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:03.149045 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:03.149384 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:03.149654 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 20:25:03.149877 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":300,"threads_started":1}
W20260806 20:25:03.150974 20193 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55732
W20260806 20:25:03.151674 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20053: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:25:03.151800 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 20053
I20260806 20:25:03.184716 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 19653
I20260806 20:25:03.186169 19728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:03.313002 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 19653
2026-08-06T20:25:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8983 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:03 dist-test-slave-6k22 krb5kdc[20392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:03 dist-test-slave-6k22 krb5kdc[20392](info): set up 2 sockets
Aug 06 20:25:03 dist-test-slave-6k22 krb5kdc[20392](info): commencing operation
krb5kdc: starting...
W20260806 20:25:04.512181 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:04 dist-test-slave-6k22 krb5kdc[20392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:25:04.743058 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:39567
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:39567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:05.071949 20406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:05.072252 20406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:05.072403 20406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:05.079821 20406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:05.079953 20406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:05.080039 20406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:05.080132 20406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:05.080196 20406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:05.090337 20406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:39567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:39567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:05.091850 20406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:05.093173 20406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:05.105623 20412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:05.105593 20415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:05.105528 20413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:05.106530 20406 server_base.cc:1034] running on GCE node
I20260806 20:25:05.107515 20406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:05.109256 20406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:05.110509 20406 hybrid_clock.cc:648] HybridClock initialized: now 1786047905110442 us; error 54 us; skew 500 ppm
I20260806 20:25:05.110993 20406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:05 dist-test-slave-6k22 krb5kdc[20392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:05.114598 20406 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:25:05.116556 20406 webserver.cc:460] Webserver started at http://127.17.109.254:40437/ using document root <none> and password file <none>
I20260806 20:25:05.117379 20406 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:05.117542 20406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:05.117940 20406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:05.121541 20406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "e5fcc04e4bfa407c909a289fe58e8d7f"
format_stamp: "Formatted at 2026-08-06 20:25:05 on dist-test-slave-6k22"
server_key: "24a583e45e2fbd8541c67050c563134b"
server_key_iv: "dd57e64e25591234d551e21a6ed06fb3"
server_key_version: "encryptionkey@0"
I20260806 20:25:05.122560 20406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "e5fcc04e4bfa407c909a289fe58e8d7f"
format_stamp: "Formatted at 2026-08-06 20:25:05 on dist-test-slave-6k22"
server_key: "24a583e45e2fbd8541c67050c563134b"
server_key_iv: "dd57e64e25591234d551e21a6ed06fb3"
server_key_version: "encryptionkey@0"
I20260806 20:25:05.128748 20406 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 20:25:05.132596 20422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:05.133989 20406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:25:05.134152 20406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "e5fcc04e4bfa407c909a289fe58e8d7f"
format_stamp: "Formatted at 2026-08-06 20:25:05 on dist-test-slave-6k22"
server_key: "24a583e45e2fbd8541c67050c563134b"
server_key_iv: "dd57e64e25591234d551e21a6ed06fb3"
server_key_version: "encryptionkey@0"
I20260806 20:25:05.134349 20406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:05.155727 20406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:05.160207 20406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:05.160521 20406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:05.179785 20406 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:39567
I20260806 20:25:05.179891 20474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:39567 every 8 connection(s)
I20260806 20:25:05.181208 20406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:05.185216 20475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:05.185577 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 20406
I20260806 20:25:05.185806 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:05.186682 17847 external_mini_cluster.cc:1428] Setting key 0e8fa9ce740597af6bec5a7aef493961
I20260806 20:25:05.200227 20475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f: Bootstrap starting.
Aug 06 20:25:05 dist-test-slave-6k22 krb5kdc[20392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:05.203838 20475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:05.204916 20475 log.cc:824] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:05.207813 20475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f: No bootstrap required, opened a new log
I20260806 20:25:05.212870 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:05.188926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.17.109.254:39567)
0806 20:25:05.189773 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:05.189785 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:05.189825 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:25:05.191968 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:05.191994 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:05.192401 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:05.192754 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:05.194672 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:05.196077 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:05.197934 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:05.198591 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:05.198772 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:05.203848 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:05.203858 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:05.203874 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:05.203931 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:05.207829 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:05.208757 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:05.208765 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:05.208771 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:05.208973 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:05.209454 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:05.209466 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:05.209470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:05.210129 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:05.210503 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:05.211098 (+ 595us) server_negotiation.cc:299] Negotiation successful
0806 20:25:05.211659 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":424,"threads_started":1}
I20260806 20:25:05.216359 20475 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5fcc04e4bfa407c909a289fe58e8d7f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39567 } }
I20260806 20:25:05.216719 20475 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:05.216825 20475 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5fcc04e4bfa407c909a289fe58e8d7f, State: Initialized, Role: FOLLOWER
I20260806 20:25:05.217343 20475 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5fcc04e4bfa407c909a289fe58e8d7f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39567 } }
I20260806 20:25:05.217592 20475 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:05.217736 20475 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:05.217897 20475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:05.220214 20475 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5fcc04e4bfa407c909a289fe58e8d7f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39567 } }
I20260806 20:25:05.220721 20475 leader_election.cc:302] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5fcc04e4bfa407c909a289fe58e8d7f; no voters:
I20260806 20:25:05.221498 20475 leader_election.cc:288] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:05.221607 20481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:05.222285 20481 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [term 1 LEADER]: Becoming Leader. State: Replica: e5fcc04e4bfa407c909a289fe58e8d7f, State: Running, Role: LEADER
I20260806 20:25:05.222908 20481 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5fcc04e4bfa407c909a289fe58e8d7f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39567 } }
I20260806 20:25:05.223915 20475 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:05.225672 20483 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5fcc04e4bfa407c909a289fe58e8d7f. Latest consensus state: current_term: 1 leader_uuid: "e5fcc04e4bfa407c909a289fe58e8d7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5fcc04e4bfa407c909a289fe58e8d7f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39567 } } }
I20260806 20:25:05.226123 20483 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:05.226110 20482 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5fcc04e4bfa407c909a289fe58e8d7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5fcc04e4bfa407c909a289fe58e8d7f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39567 } } }
I20260806 20:25:05.226613 20482 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:05.229995 20486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:05.235963 20486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:05.246016 20486 catalog_manager.cc:1347] Generated new cluster ID: 23f4d7257f724761a93ea3b62d2645c9
I20260806 20:25:05.246145 20486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:05.260823 20486 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:05.262769 20486 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:05.281178 20486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5fcc04e4bfa407c909a289fe58e8d7f: Generated new TSK 0
I20260806 20:25:05.282299 20486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:25:05.336390 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39567
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:05.669997 20503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:05.670291 20503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:05.670459 20503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:05.678130 20503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:05.678253 20503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:05.678444 20503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:05.688199 20503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:05.689489 20503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:05.690845 20503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:05.705104 20508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:05.705153 20509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:05.705176 20511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:05.705715 20503 server_base.cc:1034] running on GCE node
I20260806 20:25:05.706387 20503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:05.707345 20503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:05.708556 20503 hybrid_clock.cc:648] HybridClock initialized: now 1786047905708482 us; error 65 us; skew 500 ppm
I20260806 20:25:05.708884 20503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:05 dist-test-slave-6k22 krb5kdc[20392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:05.712253 20503 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:25:05.713781 20503 webserver.cc:460] Webserver started at http://127.17.109.193:34417/ using document root <none> and password file <none>
I20260806 20:25:05.714541 20503 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:05.714699 20503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:05.715018 20503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:05.718065 20503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "cbd41b6805a1402e95a3c2450cf09c4c"
format_stamp: "Formatted at 2026-08-06 20:25:05 on dist-test-slave-6k22"
server_key: "bed51907c1ad26028bcb15469d715d8d"
server_key_iv: "1135b7db1365370af6cc1144fe618546"
server_key_version: "encryptionkey@0"
I20260806 20:25:05.718875 20503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "cbd41b6805a1402e95a3c2450cf09c4c"
format_stamp: "Formatted at 2026-08-06 20:25:05 on dist-test-slave-6k22"
server_key: "bed51907c1ad26028bcb15469d715d8d"
server_key_iv: "1135b7db1365370af6cc1144fe618546"
server_key_version: "encryptionkey@0"
I20260806 20:25:05.724282 20503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 20:25:05.728030 20518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:05.729399 20503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:05.729581 20503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "cbd41b6805a1402e95a3c2450cf09c4c"
format_stamp: "Formatted at 2026-08-06 20:25:05 on dist-test-slave-6k22"
server_key: "bed51907c1ad26028bcb15469d715d8d"
server_key_iv: "1135b7db1365370af6cc1144fe618546"
server_key_version: "encryptionkey@0"
I20260806 20:25:05.729797 20503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:05.746649 20503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:05.750734 20503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:05.751035 20503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:05.752082 20503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:05.753435 20503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:05.753551 20503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:05.753705 20503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:05.753782 20503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:05.790290 20503 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:36013
I20260806 20:25:05.790325 20635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:36013 every 8 connection(s)
I20260806 20:25:05.791654 20503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:05.799687 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 20503
I20260806 20:25:05.799899 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:05.800583 17847 external_mini_cluster.cc:1428] Setting key 94ff332deb870c28a1e13f6cb75b77a7
Aug 06 20:25:05 dist-test-slave-6k22 krb5kdc[20392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047905, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:05.812976 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:05.794263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:41967 (local address 127.17.109.254:39567)
0806 20:25:05.794726 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:05.794730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:05.795817 (+ 1087us) server_negotiation.cc:407] Connection header received
0806 20:25:05.797646 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:05.797652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:05.797740 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:05.797862 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:05.799473 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:05.800423 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:05.802406 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:05.802884 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:05.802989 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:05.807159 (+ 4170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:05.807164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:05.807167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:05.807214 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:05.809864 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:05.810602 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:05.810607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:05.810610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:05.810739 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:05.811179 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:05.811185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:05.811188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:05.811451 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:05.811633 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:05.812380 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 20:25:05.812600 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":226,"threads_started":1}
I20260806 20:25:05.813974 20640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:05.794887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:39567 (local address 127.17.109.193:41967)
0806 20:25:05.795650 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:05.795703 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:05.797360 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:05.798111 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:05.798118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:05.798630 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:05.799265 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:05.799293 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:05.800600 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:05.800608 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:05.802259 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:05.802275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:05.803121 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:05.803130 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:05.803456 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:05.804186 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:05.804221 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:05.806932 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:05.810013 (+ 3081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:05.810023 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:05.810039 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:05.810469 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:05.810853 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:05.810859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:05.810862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:05.811054 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:05.811604 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:05.811613 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:05.811972 (+ 359us) client_negotiation.cc:769] Sending connection context
0806 20:25:05.812268 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 20:25:05.812817 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":265,"threads_started":1}
I20260806 20:25:05.815667 20636 heartbeater.cc:347] Connected to a master server at 127.17.109.254:39567
I20260806 20:25:05.815990 20636 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:05.816947 20636 heartbeater.cc:511] Master 127.17.109.254:39567 requested a full tablet report, sending...
I20260806 20:25:05.819484 20439 ts_manager.cc:194] Registered new tserver with Master: cbd41b6805a1402e95a3c2450cf09c4c (127.17.109.193:36013)
I20260806 20:25:05.821813 20439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:41967
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:25:05.856134 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39567
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:06.201983 20643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:06.202272 20643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:06.202438 20643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:06.209829 20643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:06.209941 20643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:06.210182 20643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:06.220157 20643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:06.221458 20643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:06.222829 20643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:06.237237 20648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:06.237198 20649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:06.237569 20651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:06.238442 20643 server_base.cc:1034] running on GCE node
I20260806 20:25:06.239063 20643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:06.240120 20643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:06.241364 20643 hybrid_clock.cc:648] HybridClock initialized: now 1786047906241311 us; error 45 us; skew 500 ppm
I20260806 20:25:06.241698 20643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:06 dist-test-slave-6k22 krb5kdc[20392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:06.245117 20643 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:25:06.246929 20643 webserver.cc:460] Webserver started at http://127.17.109.194:43687/ using document root <none> and password file <none>
I20260806 20:25:06.247870 20643 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:06.248056 20643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:06.248428 20643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:06.251928 20643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "be09ae662f2047d88ca9ef11912b19b7"
format_stamp: "Formatted at 2026-08-06 20:25:06 on dist-test-slave-6k22"
server_key: "f862a0b5623b87f9fb33578946b04448"
server_key_iv: "90884aa39e809454743eccfa73e34504"
server_key_version: "encryptionkey@0"
I20260806 20:25:06.252909 20643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "be09ae662f2047d88ca9ef11912b19b7"
format_stamp: "Formatted at 2026-08-06 20:25:06 on dist-test-slave-6k22"
server_key: "f862a0b5623b87f9fb33578946b04448"
server_key_iv: "90884aa39e809454743eccfa73e34504"
server_key_version: "encryptionkey@0"
I20260806 20:25:06.259027 20643 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 20:25:06.263334 20659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:06.264840 20643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:06.265034 20643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "be09ae662f2047d88ca9ef11912b19b7"
format_stamp: "Formatted at 2026-08-06 20:25:06 on dist-test-slave-6k22"
server_key: "f862a0b5623b87f9fb33578946b04448"
server_key_iv: "90884aa39e809454743eccfa73e34504"
server_key_version: "encryptionkey@0"
I20260806 20:25:06.265304 20643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:06.279479 20643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:06.284104 20643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:06.284377 20643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:06.285331 20643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:06.286655 20643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:06.286729 20643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:06.286813 20643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:06.286867 20643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:06.322016 20643 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:36273
I20260806 20:25:06.322052 20782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:36273 every 8 connection(s)
I20260806 20:25:06.323344 20643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:06.330279 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 20643
I20260806 20:25:06.330444 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:06.330960 17847 external_mini_cluster.cc:1428] Setting key d2488a9f4811add3d1197da36c9a6e62
Aug 06 20:25:06 dist-test-slave-6k22 krb5kdc[20392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:06.346390 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:06.326345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:34167 (local address 127.17.109.254:39567)
0806 20:25:06.326810 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:06.326817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:06.327972 (+ 1155us) server_negotiation.cc:407] Connection header received
0806 20:25:06.329826 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:06.329833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:06.329927 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:06.330065 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:06.331938 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:06.332841 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:06.334642 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:06.335469 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:06.335645 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:06.339749 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:06.339758 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:06.339766 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:06.339813 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:06.342716 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:06.343817 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:06.343821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:06.343823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:06.343958 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:06.344525 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:06.344529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:06.344531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:06.344837 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:06.344955 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:06.345932 (+ 977us) server_negotiation.cc:299] Negotiation successful
0806 20:25:06.346108 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":217,"threads_started":1}
I20260806 20:25:06.348065 20786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:06.326983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:39567 (local address 127.17.109.194:34167)
0806 20:25:06.327801 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:06.327847 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:06.329534 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:06.330294 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:06.330302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:06.330934 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:06.331736 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:06.331766 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:06.333027 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:06.333032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:06.334481 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:06.334494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:06.335779 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:06.335785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:06.335991 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:06.336753 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:06.336778 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:06.339472 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:06.342882 (+ 3410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:06.342894 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:06.342915 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:06.343594 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:06.344083 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:06.344091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:06.344097 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:06.344373 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:06.344985 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:06.344997 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:06.345494 (+ 497us) client_negotiation.cc:769] Sending connection context
0806 20:25:06.345812 (+ 318us) client_negotiation.cc:240] Negotiation successful
0806 20:25:06.346431 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":306,"threads_started":1}
I20260806 20:25:06.349676 20783 heartbeater.cc:347] Connected to a master server at 127.17.109.254:39567
I20260806 20:25:06.350008 20783 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:06.350766 20783 heartbeater.cc:511] Master 127.17.109.254:39567 requested a full tablet report, sending...
I20260806 20:25:06.352527 20439 ts_manager.cc:194] Registered new tserver with Master: be09ae662f2047d88ca9ef11912b19b7 (127.17.109.194:36273)
I20260806 20:25:06.353838 20439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:34167
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:25:06.387042 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39567
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:06.720368 20790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:06.720700 20790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:06.720886 20790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:06.728528 20790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:06.728698 20790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:06.728886 20790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:06.739467 20790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:06.741101 20790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:06.742653 20790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:06.757674 20795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:06.757580 20796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:06.757596 20798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:06.758626 20790 server_base.cc:1034] running on GCE node
I20260806 20:25:06.759215 20790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:06.760176 20790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:06.761380 20790 hybrid_clock.cc:648] HybridClock initialized: now 1786047906761317 us; error 61 us; skew 500 ppm
I20260806 20:25:06.761700 20790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:06 dist-test-slave-6k22 krb5kdc[20392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:06.765118 20790 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:25:06.766630 20790 webserver.cc:460] Webserver started at http://127.17.109.195:35337/ using document root <none> and password file <none>
I20260806 20:25:06.767323 20790 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:06.767477 20790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:06.767831 20790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:06.770906 20790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "8cfd3d455052483fac8315c2b3d19a27"
format_stamp: "Formatted at 2026-08-06 20:25:06 on dist-test-slave-6k22"
server_key: "6ed76d91cd75f24eb91f1daef01b8ade"
server_key_iv: "38c83f259b1ee0a7555ab177ff3caa2c"
server_key_version: "encryptionkey@0"
I20260806 20:25:06.771759 20790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "8cfd3d455052483fac8315c2b3d19a27"
format_stamp: "Formatted at 2026-08-06 20:25:06 on dist-test-slave-6k22"
server_key: "6ed76d91cd75f24eb91f1daef01b8ade"
server_key_iv: "38c83f259b1ee0a7555ab177ff3caa2c"
server_key_version: "encryptionkey@0"
I20260806 20:25:06.776986 20790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:25:06.780577 20805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:06.782016 20790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:25:06.782204 20790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "8cfd3d455052483fac8315c2b3d19a27"
format_stamp: "Formatted at 2026-08-06 20:25:06 on dist-test-slave-6k22"
server_key: "6ed76d91cd75f24eb91f1daef01b8ade"
server_key_iv: "38c83f259b1ee0a7555ab177ff3caa2c"
server_key_version: "encryptionkey@0"
I20260806 20:25:06.782418 20790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:06.818750 20790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:06.824131 20790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:06.824442 20790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:06.825539 20636 heartbeater.cc:503] Master 127.17.109.254:39567 was elected leader, sending a full tablet report...
I20260806 20:25:06.825704 20790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:06.827237 20790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:06.827356 20790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:06.827521 20790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:06.827612 20790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:06.862918 20790 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:44045
I20260806 20:25:06.862968 20923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:44045 every 8 connection(s)
I20260806 20:25:06.864281 20790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:06.870427 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 20790
I20260806 20:25:06.870622 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:06.871232 17847 external_mini_cluster.cc:1428] Setting key 44fd47bbe75fd86493353784da31a0f4
Aug 06 20:25:06 dist-test-slave-6k22 krb5kdc[20392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047906, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:06.885726 20926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:06.866952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:52297 (local address 127.17.109.254:39567)
0806 20:25:06.867449 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:06.867457 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:06.868499 (+ 1042us) server_negotiation.cc:407] Connection header received
0806 20:25:06.870692 (+ 2193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:06.870700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:06.870785 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:06.870883 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:06.872931 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:06.874132 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:06.876014 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:06.876408 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:06.876503 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:06.879919 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:06.879924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:06.879927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:06.879958 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:06.882645 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:06.883329 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:06.883333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:06.883335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:06.883455 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:06.883884 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:06.883887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:06.883889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:06.884138 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:06.884257 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:06.885051 (+ 794us) server_negotiation.cc:299] Negotiation successful
0806 20:25:06.885289 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":222,"threads_started":1}
I20260806 20:25:06.886812 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:06.867525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:39567 (local address 127.17.109.195:52297)
0806 20:25:06.868284 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:06.868361 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:06.870269 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:06.871158 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:06.871169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:06.871934 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:06.872748 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:06.872777 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:06.874291 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:06.874295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:06.875864 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:06.875883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:06.876603 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:06.876608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:06.876857 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:06.877495 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:06.877521 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:06.879721 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:06.882796 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:06.882802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:06.882813 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:06.883217 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:06.883544 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:06.883547 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:06.883549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:06.883777 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:06.884257 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:06.884263 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:06.884559 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 20:25:06.884913 (+ 354us) client_negotiation.cc:240] Negotiation successful
0806 20:25:06.885407 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":276,"threads_started":1}
I20260806 20:25:06.888182 20924 heartbeater.cc:347] Connected to a master server at 127.17.109.254:39567
I20260806 20:25:06.888484 20924 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:06.889218 20924 heartbeater.cc:511] Master 127.17.109.254:39567 requested a full tablet report, sending...
I20260806 20:25:06.890602 20439 ts_manager.cc:194] Registered new tserver with Master: 8cfd3d455052483fac8315c2b3d19a27 (127.17.109.195:44045)
I20260806 20:25:06.891394 20439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:52297
I20260806 20:25:06.899310 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 20:25:06.935915 20926 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:06.932987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60334 (local address 127.17.109.254:39567)
0806 20:25:06.933312 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:06.933320 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:06.933704 (+ 384us) server_negotiation.cc:407] Connection header received
0806 20:25:06.933827 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:06.933831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:06.933925 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:06.934028 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:06.935648 (+ 1620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60334: BlockingRecv error: recv got EOF from 127.0.0.1:60334 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:25:06.940102 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20503
W20260806 20:25:06.943022 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20503: Not authorized: Client connection negotiation failed: client connection to 127.17.109.193:36013: server requires authentication, but client does not have Kerberos credentials available
I20260806 20:25:06.943216 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 20503
I20260806 20:25:06.960740 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20643
W20260806 20:25:06.963666 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20643: Not authorized: Client connection negotiation failed: client connection to 127.17.109.194:36273: server requires authentication, but client does not have Kerberos credentials available
I20260806 20:25:06.963850 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 20643
I20260806 20:25:06.983983 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20790
W20260806 20:25:06.987430 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20790: Not authorized: Client connection negotiation failed: client connection to 127.17.109.195:44045: server requires authentication, but client does not have Kerberos credentials available
I20260806 20:25:06.987668 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 20790
I20260806 20:25:07.006606 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20406
I20260806 20:25:07.008132 20470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:07.113453 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 20406
2026-08-06T20:25:07Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3795 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:07 dist-test-slave-6k22 krb5kdc[20942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:07 dist-test-slave-6k22 krb5kdc[20942](info): set up 2 sockets
Aug 06 20:25:07 dist-test-slave-6k22 krb5kdc[20942](info): commencing operation
krb5kdc: starting...
W20260806 20:25:08.311698 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:08 dist-test-slave-6k22 krb5kdc[20942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:25:08.556358 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:36671
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:36671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:08.905613 20957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:08.905989 20957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:08.906157 20957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:08.913591 20957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:08.913753 20957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:08.913869 20957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:08.913972 20957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:08.914064 20957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:08.924458 20957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:36671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:36671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:08.925861 20957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:08.927225 20957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:08.939020 20962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:08.938997 20963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:08.939230 20966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:08.939746 20957 server_base.cc:1034] running on GCE node
I20260806 20:25:08.940500 20957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:08.941838 20957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:08.943081 20957 hybrid_clock.cc:648] HybridClock initialized: now 1786047908943004 us; error 60 us; skew 500 ppm
I20260806 20:25:08.943421 20957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:08 dist-test-slave-6k22 krb5kdc[20942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047908, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:08.953343 20957 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:25:08.954924 20957 webserver.cc:460] Webserver started at http://127.17.109.254:38019/ using document root <none> and password file <none>
I20260806 20:25:08.955703 20957 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:08.955832 20957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:08.956182 20957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:08.959445 20957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "20e1767d2c844ef0b8c7f53249ff7b73"
format_stamp: "Formatted at 2026-08-06 20:25:08 on dist-test-slave-6k22"
server_key: "85599b2d405dc3ec9f437b4990858d1c"
server_key_iv: "681ae42380aa694a86e11ac387bb6327"
server_key_version: "encryptionkey@0"
I20260806 20:25:08.960331 20957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "20e1767d2c844ef0b8c7f53249ff7b73"
format_stamp: "Formatted at 2026-08-06 20:25:08 on dist-test-slave-6k22"
server_key: "85599b2d405dc3ec9f437b4990858d1c"
server_key_iv: "681ae42380aa694a86e11ac387bb6327"
server_key_version: "encryptionkey@0"
I20260806 20:25:08.965806 20957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 20:25:08.969568 20973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:08.971084 20957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:25:08.971324 20957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "20e1767d2c844ef0b8c7f53249ff7b73"
format_stamp: "Formatted at 2026-08-06 20:25:08 on dist-test-slave-6k22"
server_key: "85599b2d405dc3ec9f437b4990858d1c"
server_key_iv: "681ae42380aa694a86e11ac387bb6327"
server_key_version: "encryptionkey@0"
I20260806 20:25:08.971552 20957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:08.984233 20957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:08.988854 20957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:08.989159 20957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:09.009840 20957 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:36671
I20260806 20:25:09.009863 21025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:36671 every 8 connection(s)
I20260806 20:25:09.011583 20957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:09.016403 21026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:09.020646 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 20957
I20260806 20:25:09.020980 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:09.021623 17847 external_mini_cluster.cc:1428] Setting key af73b1076a77e9c6b5695163baafa736
I20260806 20:25:09.028394 21026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73: Bootstrap starting.
I20260806 20:25:09.032389 21026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:09.033664 21026 log.cc:824] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:09.037482 21026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73: No bootstrap required, opened a new log
Aug 06 20:25:09 dist-test-slave-6k22 krb5kdc[20942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:09.050917 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:09.023536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.17.109.254:36671)
0806 20:25:09.024441 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:09.024453 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:09.024494 (+ 41us) server_negotiation.cc:407] Connection header received
0806 20:25:09.026646 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:09.026670 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:09.027057 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:09.027393 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:09.029513 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:09.030919 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:09.032855 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:09.033541 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:09.033671 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:09.039799 (+ 6128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:09.039812 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:09.039833 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:09.039890 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:09.045674 (+ 5784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:09.046475 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:09.046491 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:09.046502 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:09.046736 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:09.047135 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:09.047141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:09.047153 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:09.048017 (+ 864us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:09.048373 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:09.048790 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 20:25:09.049347 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":406,"threads_started":1}
I20260806 20:25:09.051187 21026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1767d2c844ef0b8c7f53249ff7b73" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 36671 } }
I20260806 20:25:09.051702 21026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:09.051826 21026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e1767d2c844ef0b8c7f53249ff7b73, State: Initialized, Role: FOLLOWER
I20260806 20:25:09.052460 21026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1767d2c844ef0b8c7f53249ff7b73" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 36671 } }
I20260806 20:25:09.052737 21026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:09.052913 21026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:09.053129 21026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:09.055794 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:25:09.056417 21026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1767d2c844ef0b8c7f53249ff7b73" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 36671 } }
I20260806 20:25:09.057039 21026 leader_election.cc:302] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20e1767d2c844ef0b8c7f53249ff7b73; no voters:
I20260806 20:25:09.057717 21026 leader_election.cc:288] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:09.057935 21032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:09.060046 21032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [term 1 LEADER]: Becoming Leader. State: Replica: 20e1767d2c844ef0b8c7f53249ff7b73, State: Running, Role: LEADER
I20260806 20:25:09.060557 21026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:09.060747 21032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1767d2c844ef0b8c7f53249ff7b73" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 36671 } }
I20260806 20:25:09.065536 21033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20e1767d2c844ef0b8c7f53249ff7b73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1767d2c844ef0b8c7f53249ff7b73" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 36671 } } }
I20260806 20:25:09.065440 21034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20e1767d2c844ef0b8c7f53249ff7b73. Latest consensus state: current_term: 1 leader_uuid: "20e1767d2c844ef0b8c7f53249ff7b73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20e1767d2c844ef0b8c7f53249ff7b73" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 36671 } } }
I20260806 20:25:09.065953 21034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:09.065953 21033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:09.066996 21038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:09.083630 21038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:09.098603 21038 catalog_manager.cc:1347] Generated new cluster ID: ba0dde77760549cf893dd7d38d24279a
I20260806 20:25:09.098749 21038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:09.110426 21038 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:09.112282 21038 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:09.119254 21038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20e1767d2c844ef0b8c7f53249ff7b73: Generated new TSK 0
I20260806 20:25:09.120350 21038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:25:09.127184 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:09.116716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.17.109.254:36671)
0806 20:25:09.116923 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:09.116931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:09.117707 (+ 776us) server_negotiation.cc:407] Connection header received
0806 20:25:09.117929 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:09.117936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:09.118020 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:09.118148 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:09.119810 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:09.120859 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:09.123029 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:09.123470 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:09.123627 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:09.125057 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:09.125062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:09.125065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:09.126102 (+ 1037us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:09.126207 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:09.126244 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:09.126381 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:09.126684 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 20:25:09.126774 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:25:09.136096 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 20957
I20260806 20:25:09.137639 21021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:09.235896 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 20957
2026-08-06T20:25:09Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2118 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:09 dist-test-slave-6k22 krb5kdc[21063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:09 dist-test-slave-6k22 krb5kdc[21063](info): set up 2 sockets
Aug 06 20:25:09 dist-test-slave-6k22 krb5kdc[21063](info): commencing operation
krb5kdc: starting...
W20260806 20:25:10.367852 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:10 dist-test-slave-6k22 krb5kdc[21063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:25:10.608431 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:37533
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:37533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:10.950007 21076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:10.950294 21076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:10.950497 21076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:10.957594 21076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:10.957700 21076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:10.957747 21076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:10.957803 21076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:10.957856 21076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:10.967764 21076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:37533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:37533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:10.969096 21076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:10.970376 21076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:10.983076 21081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:10.983026 21085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:10.983114 21082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:10.984521 21076 server_base.cc:1034] running on GCE node
I20260806 20:25:10.985412 21076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:10.986891 21076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:10.988154 21076 hybrid_clock.cc:648] HybridClock initialized: now 1786047910988115 us; error 63 us; skew 500 ppm
I20260806 20:25:10.988536 21076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:10 dist-test-slave-6k22 krb5kdc[21063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047910, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:10.991874 21076 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:25:10.993487 21076 webserver.cc:460] Webserver started at http://127.17.109.254:41889/ using document root <none> and password file <none>
I20260806 20:25:10.994138 21076 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:10.994246 21076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:10.994509 21076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:10.997699 21076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "747248b2d93345d08ed7b9e28183f6d1"
format_stamp: "Formatted at 2026-08-06 20:25:10 on dist-test-slave-6k22"
server_key: "81d8e4b55b5ce310d60af1dbecd21ba1"
server_key_iv: "1a80cac85226b60d119a5c30031f0c71"
server_key_version: "encryptionkey@0"
I20260806 20:25:10.998514 21076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "747248b2d93345d08ed7b9e28183f6d1"
format_stamp: "Formatted at 2026-08-06 20:25:10 on dist-test-slave-6k22"
server_key: "81d8e4b55b5ce310d60af1dbecd21ba1"
server_key_iv: "1a80cac85226b60d119a5c30031f0c71"
server_key_version: "encryptionkey@0"
I20260806 20:25:11.004245 21076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:25:11.008003 21092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:11.009511 21076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:25:11.009768 21076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "747248b2d93345d08ed7b9e28183f6d1"
format_stamp: "Formatted at 2026-08-06 20:25:10 on dist-test-slave-6k22"
server_key: "81d8e4b55b5ce310d60af1dbecd21ba1"
server_key_iv: "1a80cac85226b60d119a5c30031f0c71"
server_key_version: "encryptionkey@0"
I20260806 20:25:11.009991 21076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:11.035321 21076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:11.039831 21076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:11.040153 21076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:11.059592 21076 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:37533
I20260806 20:25:11.059700 21144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:37533 every 8 connection(s)
I20260806 20:25:11.061008 21076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:11.063246 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 21076
I20260806 20:25:11.063503 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:11.064352 17847 external_mini_cluster.cc:1428] Setting key abf2ce9f7176c93afc20dbf1c6f8318b
I20260806 20:25:11.065917 21145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 20:25:11 dist-test-slave-6k22 krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:11.078919 21145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1: Bootstrap starting.
I20260806 20:25:11.082788 21145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:11.084013 21145 log.cc:824] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:11.087782 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:11.066627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50140 (local address 127.17.109.254:37533)
0806 20:25:11.067487 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:11.067500 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:11.067542 (+ 42us) server_negotiation.cc:407] Connection header received
0806 20:25:11.069218 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:11.069240 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:11.069634 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:11.070133 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:11.071642 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:11.073017 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:11.074337 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:11.075006 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:11.075246 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:11.079731 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:11.079739 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:11.079756 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:11.079797 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:11.083011 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:11.083815 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:11.083835 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:11.083842 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:11.084020 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:11.084458 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:11.084467 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:11.084472 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:11.085019 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:11.085385 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:11.085949 (+ 564us) server_negotiation.cc:299] Negotiation successful
0806 20:25:11.086410 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":362,"threads_started":1}
I20260806 20:25:11.087872 21145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1: No bootstrap required, opened a new log
I20260806 20:25:11.096194 21145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "747248b2d93345d08ed7b9e28183f6d1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37533 } }
I20260806 20:25:11.096550 21145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:11.096628 21145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 747248b2d93345d08ed7b9e28183f6d1, State: Initialized, Role: FOLLOWER
I20260806 20:25:11.097198 21145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "747248b2d93345d08ed7b9e28183f6d1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37533 } }
I20260806 20:25:11.097409 21145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:11.097512 21145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:11.097657 21145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:11.100241 21145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "747248b2d93345d08ed7b9e28183f6d1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37533 } }
I20260806 20:25:11.100708 21145 leader_election.cc:302] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 747248b2d93345d08ed7b9e28183f6d1; no voters:
I20260806 20:25:11.101269 21145 leader_election.cc:288] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:11.101408 21150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:11.102176 21150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [term 1 LEADER]: Becoming Leader. State: Replica: 747248b2d93345d08ed7b9e28183f6d1, State: Running, Role: LEADER
I20260806 20:25:11.102739 21150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "747248b2d93345d08ed7b9e28183f6d1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37533 } }
I20260806 20:25:11.103516 21145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:11.105664 21151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "747248b2d93345d08ed7b9e28183f6d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "747248b2d93345d08ed7b9e28183f6d1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37533 } } }
I20260806 20:25:11.106091 21151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:11.106650 21152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 747248b2d93345d08ed7b9e28183f6d1. Latest consensus state: current_term: 1 leader_uuid: "747248b2d93345d08ed7b9e28183f6d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "747248b2d93345d08ed7b9e28183f6d1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37533 } } }
I20260806 20:25:11.107067 21152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:11.107197 21156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:11.112818 21156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:11.125818 21156 catalog_manager.cc:1347] Generated new cluster ID: 8a788ca6908d4a80aa73eaad8b3b4a4a
I20260806 20:25:11.125943 21156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:11.135983 21156 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:11.137823 21156 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:11.147818 21156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 747248b2d93345d08ed7b9e28183f6d1: Generated new TSK 0
I20260806 20:25:11.148872 21156 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:25:11.203157 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37533
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:11.544486 21171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:11.544809 21171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:11.544909 21171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:11.552016 21171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:11.552124 21171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:11.552248 21171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:11.561892 21171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:11.563145 21171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:11.564503 21171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:11.578109 21177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:11.578357 21180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:11.578357 21178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:11.579790 21171 server_base.cc:1034] running on GCE node
I20260806 20:25:11.580430 21171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:11.581548 21171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:11.582849 21171 hybrid_clock.cc:648] HybridClock initialized: now 1786047911582813 us; error 33 us; skew 500 ppm
I20260806 20:25:11.583204 21171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:11 dist-test-slave-6k22 krb5kdc[21063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:11.586967 21171 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:25:11.588460 21171 webserver.cc:460] Webserver started at http://127.17.109.193:38539/ using document root <none> and password file <none>
I20260806 20:25:11.589155 21171 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:11.589267 21171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:11.589524 21171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:11.592597 21171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "d003c1712ea54aafb75b161c5725776d"
format_stamp: "Formatted at 2026-08-06 20:25:11 on dist-test-slave-6k22"
server_key: "389709f12eba2c6ceb1737dc8b1ba9ca"
server_key_iv: "2603b7c0809d338baadb1bd522967131"
server_key_version: "encryptionkey@0"
I20260806 20:25:11.593387 21171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "d003c1712ea54aafb75b161c5725776d"
format_stamp: "Formatted at 2026-08-06 20:25:11 on dist-test-slave-6k22"
server_key: "389709f12eba2c6ceb1737dc8b1ba9ca"
server_key_iv: "2603b7c0809d338baadb1bd522967131"
server_key_version: "encryptionkey@0"
I20260806 20:25:11.598553 21171 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:25:11.601990 21188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:11.603381 21171 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:25:11.603618 21171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "d003c1712ea54aafb75b161c5725776d"
format_stamp: "Formatted at 2026-08-06 20:25:11 on dist-test-slave-6k22"
server_key: "389709f12eba2c6ceb1737dc8b1ba9ca"
server_key_iv: "2603b7c0809d338baadb1bd522967131"
server_key_version: "encryptionkey@0"
I20260806 20:25:11.603837 21171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:11.620167 21171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:11.624449 21171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:11.624769 21171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:11.625797 21171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:11.627126 21171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:11.627242 21171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:11.627417 21171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:11.627492 21171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:11.661803 21171 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:46579
I20260806 20:25:11.661854 21303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:46579 every 8 connection(s)
I20260806 20:25:11.663187 21171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:11.667212 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 21171
I20260806 20:25:11.667441 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:11.668143 17847 external_mini_cluster.cc:1428] Setting key 12bd23db04900646c13d1df6a13183e0
Aug 06 20:25:11 dist-test-slave-6k22 krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047911, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:11.684398 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:11.665905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:43439 (local address 127.17.109.254:37533)
0806 20:25:11.666435 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:11.666439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:11.667284 (+ 845us) server_negotiation.cc:407] Connection header received
0806 20:25:11.669271 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:11.669275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:11.669337 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:11.669430 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:11.671009 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:11.671821 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:11.673524 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:11.673943 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:11.674085 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:11.677927 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:11.677938 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:11.677941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:11.677975 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:11.680732 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:11.681727 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:11.681731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:11.681733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:11.681866 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:11.682449 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:11.682452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:11.682454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:11.682749 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:11.682874 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:11.683903 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0806 20:25:11.684101 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":299,"threads_started":1}
I20260806 20:25:11.685972 21307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:11.666330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:37533 (local address 127.17.109.193:43439)
0806 20:25:11.667078 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:11.667118 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:11.668978 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:11.669707 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:11.669718 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:11.670285 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:11.670836 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:11.670853 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:11.672000 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:11.672004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:11.673369 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:11.673382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:11.674197 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:11.674204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:11.674494 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:11.675209 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:11.675231 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:11.677635 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:11.680884 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:11.680895 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:11.680915 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:11.681596 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:11.682000 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:11.682008 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:11.682014 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:11.682279 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:11.682892 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:11.682904 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:11.683422 (+ 518us) client_negotiation.cc:769] Sending connection context
0806 20:25:11.683790 (+ 368us) client_negotiation.cc:240] Negotiation successful
0806 20:25:11.684400 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":310,"threads_started":1}
I20260806 20:25:11.688140 21304 heartbeater.cc:347] Connected to a master server at 127.17.109.254:37533
I20260806 20:25:11.688476 21304 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:11.689293 21304 heartbeater.cc:511] Master 127.17.109.254:37533 requested a full tablet report, sending...
I20260806 20:25:11.691926 21110 ts_manager.cc:194] Registered new tserver with Master: d003c1712ea54aafb75b161c5725776d (127.17.109.193:46579)
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
I20260806 20:25:11.694037 21110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:43439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:25:11.727113 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37533
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:12.066365 21311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:12.066722 21311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:12.066921 21311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:12.074774 21311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:12.074949 21311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:12.075162 21311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:12.085438 21311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:12.086858 21311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:12.088435 21311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:12.102306 21319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:12.102365 21317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:12.102397 21316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:12.105111 21311 server_base.cc:1034] running on GCE node
I20260806 20:25:12.105786 21311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:12.106781 21311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:12.108016 21311 hybrid_clock.cc:648] HybridClock initialized: now 1786047912108020 us; error 107 us; skew 500 ppm
I20260806 20:25:12.108361 21311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:12.111985 21311 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:25:12.113622 21311 webserver.cc:460] Webserver started at http://127.17.109.194:41305/ using document root <none> and password file <none>
I20260806 20:25:12.114416 21311 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:12.114562 21311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:12.114886 21311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:12.117955 21311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "8897055fdf16483db81ff4397dc1e044"
format_stamp: "Formatted at 2026-08-06 20:25:12 on dist-test-slave-6k22"
server_key: "e507aed7e906c98b21584dc520f53e45"
server_key_iv: "2e5565a407ad412b1066e74e9502055e"
server_key_version: "encryptionkey@0"
I20260806 20:25:12.118811 21311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "8897055fdf16483db81ff4397dc1e044"
format_stamp: "Formatted at 2026-08-06 20:25:12 on dist-test-slave-6k22"
server_key: "e507aed7e906c98b21584dc520f53e45"
server_key_iv: "2e5565a407ad412b1066e74e9502055e"
server_key_version: "encryptionkey@0"
I20260806 20:25:12.124317 21311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:25:12.128087 21326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:12.129592 21311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:12.129822 21311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "8897055fdf16483db81ff4397dc1e044"
format_stamp: "Formatted at 2026-08-06 20:25:12 on dist-test-slave-6k22"
server_key: "e507aed7e906c98b21584dc520f53e45"
server_key_iv: "2e5565a407ad412b1066e74e9502055e"
server_key_version: "encryptionkey@0"
I20260806 20:25:12.130045 21311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:12.143839 21311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:12.148005 21311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:12.148342 21311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:12.149319 21311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:12.150652 21311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:12.150736 21311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:12.150820 21311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:12.150869 21311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:12.191116 21311 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:38039
I20260806 20:25:12.191140 21440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:38039 every 8 connection(s)
I20260806 20:25:12.192487 21311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:12.201457 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 21311
I20260806 20:25:12.201694 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:12.202246 17847 external_mini_cluster.cc:1428] Setting key cf2d84fdc32ce3a10b7267ef0adf146f
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:12.216320 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:12.195257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:58519 (local address 127.17.109.254:37533)
0806 20:25:12.195782 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:12.195790 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:12.196768 (+ 978us) server_negotiation.cc:407] Connection header received
0806 20:25:12.198665 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:12.198671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:12.198749 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:12.198877 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:12.200442 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.201537 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.203414 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.204210 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.204332 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:12.208172 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:12.208183 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:12.208191 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:12.208242 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:12.212717 (+ 4475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.213559 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.213566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.213572 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.213788 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.214274 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.214280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.214286 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.214676 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:12.214821 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:12.215831 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0806 20:25:12.216015 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":388,"thread_start_us":221,"threads_started":1}
I20260806 20:25:12.217901 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:12.195862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:37533 (local address 127.17.109.194:58519)
0806 20:25:12.196601 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:12.196639 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:12.198329 (+ 1690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:12.199065 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:12.199072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:12.199679 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:12.200259 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:12.200277 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.201772 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.201778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:12.203253 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:12.203268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.204474 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.204483 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:12.204820 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:12.205590 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:12.205612 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:12.207927 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:12.212896 (+ 4969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:12.212903 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:12.212916 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:12.213374 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:12.213909 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:12.213912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:12.213915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:12.214113 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:12.214817 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:12.214827 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:12.215339 (+ 512us) client_negotiation.cc:769] Sending connection context
0806 20:25:12.215660 (+ 321us) client_negotiation.cc:240] Negotiation successful
0806 20:25:12.216271 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":298,"threads_started":1}
I20260806 20:25:12.219504 21441 heartbeater.cc:347] Connected to a master server at 127.17.109.254:37533
I20260806 20:25:12.219930 21441 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:12.220748 21441 heartbeater.cc:511] Master 127.17.109.254:37533 requested a full tablet report, sending...
I20260806 20:25:12.222414 21110 ts_manager.cc:194] Registered new tserver with Master: 8897055fdf16483db81ff4397dc1e044 (127.17.109.194:38039)
I20260806 20:25:12.223429 21110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:58519
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:25:12.260468 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37533
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:12.597062 21448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:12.597329 21448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:12.597491 21448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:12.604887 21448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:12.604998 21448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:12.605203 21448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:12.615795 21448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:12.617098 21448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:12.618446 21448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:12.632258 21453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:12.632162 21454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:12.632206 21456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:12.633925 21448 server_base.cc:1034] running on GCE node
I20260806 20:25:12.634642 21448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:12.635803 21448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:12.637046 21448 hybrid_clock.cc:648] HybridClock initialized: now 1786047912637003 us; error 28 us; skew 500 ppm
I20260806 20:25:12.637385 21448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:12.641004 21448 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:25:12.642656 21448 webserver.cc:460] Webserver started at http://127.17.109.195:45019/ using document root <none> and password file <none>
I20260806 20:25:12.643389 21448 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:12.643544 21448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:12.643918 21448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:12.646906 21448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "cb0e9484f6d943fda7dd4ffd8037a895"
format_stamp: "Formatted at 2026-08-06 20:25:12 on dist-test-slave-6k22"
server_key: "5b416aba74edfcacbf3ea0fabfa397c3"
server_key_iv: "2cacd25f28c2593cc21453ef2f580594"
server_key_version: "encryptionkey@0"
I20260806 20:25:12.647778 21448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "cb0e9484f6d943fda7dd4ffd8037a895"
format_stamp: "Formatted at 2026-08-06 20:25:12 on dist-test-slave-6k22"
server_key: "5b416aba74edfcacbf3ea0fabfa397c3"
server_key_iv: "2cacd25f28c2593cc21453ef2f580594"
server_key_version: "encryptionkey@0"
I20260806 20:25:12.653254 21448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:25:12.657040 21463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:12.658571 21448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:12.658828 21448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "cb0e9484f6d943fda7dd4ffd8037a895"
format_stamp: "Formatted at 2026-08-06 20:25:12 on dist-test-slave-6k22"
server_key: "5b416aba74edfcacbf3ea0fabfa397c3"
server_key_iv: "2cacd25f28c2593cc21453ef2f580594"
server_key_version: "encryptionkey@0"
I20260806 20:25:12.659053 21448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:12.677551 21448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:12.682276 21448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:12.682611 21448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:12.683773 21448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:12.685202 21448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:12.685322 21448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:12.685477 21448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:12.685557 21448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:12.697598 21304 heartbeater.cc:503] Master 127.17.109.254:37533 was elected leader, sending a full tablet report...
I20260806 20:25:12.722754 21448 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:35513
I20260806 20:25:12.722860 21577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:35513 every 8 connection(s)
I20260806 20:25:12.724188 21448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:12.724620 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 21448
I20260806 20:25:12.724865 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:12.725515 17847 external_mini_cluster.cc:1428] Setting key 716b40905ec7d68695148ad09589bde9
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:12.748116 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:12.727518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:49307 (local address 127.17.109.254:37533)
0806 20:25:12.728194 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:12.728200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:12.729572 (+ 1372us) server_negotiation.cc:407] Connection header received
0806 20:25:12.731503 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:12.731509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:12.731629 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:12.731781 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:12.733527 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.734464 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.736226 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.736680 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.736844 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:12.741028 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:12.741036 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:12.741041 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:12.741086 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:12.744520 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.745487 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.745491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.745493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.745622 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.746197 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.746202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.746232 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.746598 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:12.746749 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:12.747514 (+ 765us) server_negotiation.cc:299] Negotiation successful
0806 20:25:12.747816 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":363,"threads_started":1}
I20260806 20:25:12.749156 21581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:12.728370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:37533 (local address 127.17.109.195:49307)
0806 20:25:12.729382 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:12.729445 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:12.731219 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:12.732035 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:12.732045 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:12.732599 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:12.733334 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:12.733355 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.734645 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.734649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:12.736060 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:12.736073 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.736938 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.736946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:12.737212 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:12.737993 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:12.738018 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:12.740661 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:12.744758 (+ 4097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:12.744766 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:12.744781 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:12.745338 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:12.745798 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:12.745803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:12.745805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:12.746034 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:12.746756 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:12.746763 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:12.747111 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 20:25:12.747363 (+ 252us) client_negotiation.cc:240] Negotiation successful
0806 20:25:12.747893 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":401,"threads_started":1}
I20260806 20:25:12.750785 21578 heartbeater.cc:347] Connected to a master server at 127.17.109.254:37533
I20260806 20:25:12.751173 21578 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:12.752122 21578 heartbeater.cc:511] Master 127.17.109.254:37533 requested a full tablet report, sending...
I20260806 20:25:12.754053 21110 ts_manager.cc:194] Registered new tserver with Master: cb0e9484f6d943fda7dd4ffd8037a895 (127.17.109.195:35513)
I20260806 20:25:12.755350 21110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:49307
I20260806 20:25:12.755483 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:12.811081 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:12.796342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50156 (local address 127.17.109.254:37533)
0806 20:25:12.796623 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:12.796629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:12.796924 (+ 295us) server_negotiation.cc:407] Connection header received
0806 20:25:12.797077 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:12.797081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:12.797147 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:12.797261 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:12.798619 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.799398 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.800673 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.801146 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.801264 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:12.805342 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:12.805347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:12.805350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:12.805383 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:12.808216 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.809005 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.809012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.809015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.809150 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.809552 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.809557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.809560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.809909 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:12.810240 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:12.810529 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 20:25:12.810757 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 20:25:12.811843 21110 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50156
I20260806 20:25:12.813298 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21171
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:25:12.830617 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:12.814760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.17.109.193:46579)
0806 20:25:12.815488 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:12.815494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:12.815519 (+ 25us) server_negotiation.cc:407] Connection header received
0806 20:25:12.815625 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:12.815632 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:12.815869 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:12.815997 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:12.817210 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.818249 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.819444 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.820272 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.820376 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:12.824469 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:12.824481 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:12.824495 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:12.824527 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:12.827832 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.828519 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.828527 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.828530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.828694 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.829106 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.829119 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.829121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.829483 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:12.829712 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:12.829995 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 20:25:12.830236 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":407,"threads_started":1}
W20260806 20:25:12.831984 21299 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58890
W20260806 20:25:12.833169 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21171: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:25:12.833411 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 21171
I20260806 20:25:12.852011 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21311
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:25:12.868497 21591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:12.853148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.17.109.194:38039)
0806 20:25:12.853805 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:12.853812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:12.853839 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:25:12.853900 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:12.853907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:12.854130 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:12.854273 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:12.855497 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.856627 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.857894 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.858460 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.858556 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:12.862918 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:12.862929 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:12.862944 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:12.862977 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:12.865975 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.866629 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.866636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.866639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.866771 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.867095 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.867099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.867101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.867435 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:12.867848 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:12.868010 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 20:25:12.868217 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":345,"threads_started":1}
W20260806 20:25:12.869781 21436 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40156
W20260806 20:25:12.870965 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21311: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:25:12.871158 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 21311
I20260806 20:25:12.888872 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21448
Aug 06 20:25:12 dist-test-slave-6k22 krb5kdc[21063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047912, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:25:12.907231 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:12.890012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35832 (local address 127.17.109.195:35513)
0806 20:25:12.890680 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:12.890689 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:12.890723 (+ 34us) server_negotiation.cc:407] Connection header received
0806 20:25:12.891001 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:12.891013 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:12.891257 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:12.891406 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:12.892876 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.893963 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.895123 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:12.895782 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:12.895883 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:12.900742 (+ 4859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:12.900756 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:12.900771 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:12.900815 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:12.904334 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.904986 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.905019 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.905025 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.905178 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:12.905546 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:12.905549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:12.905552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:12.905935 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:12.906178 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:12.906507 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 20:25:12.906753 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":365,"threads_started":1}
W20260806 20:25:12.908676 21573 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35832
W20260806 20:25:12.910106 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21448: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:25:12.910341 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 21448
I20260806 20:25:12.929330 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21076
I20260806 20:25:12.930606 21140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:13.036448 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 21076
2026-08-06T20:25:13Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3804 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:13 dist-test-slave-6k22 krb5kdc[21596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:13 dist-test-slave-6k22 krb5kdc[21596](info): set up 2 sockets
Aug 06 20:25:13 dist-test-slave-6k22 krb5kdc[21596](info): commencing operation
krb5kdc: starting...
W20260806 20:25:14.209476 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:14 dist-test-slave-6k22 krb5kdc[21596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:25:14.457283 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34525
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34525
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:14.807520 21609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:14.807946 21609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:14.808130 21609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:14.816375 21609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:14.816504 21609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:14.816603 21609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:14.816725 21609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:14.816790 21609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:14.828303 21609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:14.829711 21609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:14.831060 21609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:14.843549 21614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:14.843750 21617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:14.843731 21615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:14.844732 21609 server_base.cc:1034] running on GCE node
I20260806 20:25:14.845655 21609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:14.847301 21609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:14.848603 21609 hybrid_clock.cc:648] HybridClock initialized: now 1786047914848522 us; error 78 us; skew 500 ppm
I20260806 20:25:14.849054 21609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:14 dist-test-slave-6k22 krb5kdc[21596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047914, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:14.852663 21609 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:25:14.854357 21609 webserver.cc:460] Webserver started at http://127.17.109.254:34967/ using document root <none> and password file <none>
I20260806 20:25:14.855089 21609 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:14.855203 21609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:14.855497 21609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:14.858979 21609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "3368842aa71e46c3b4396f5f330c1361"
format_stamp: "Formatted at 2026-08-06 20:25:14 on dist-test-slave-6k22"
server_key: "75bd941a2efee36668efc711c5afea05"
server_key_iv: "5216edb8c69da3a25c83c8a0736d6ad9"
server_key_version: "encryptionkey@0"
I20260806 20:25:14.859941 21609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "3368842aa71e46c3b4396f5f330c1361"
format_stamp: "Formatted at 2026-08-06 20:25:14 on dist-test-slave-6k22"
server_key: "75bd941a2efee36668efc711c5afea05"
server_key_iv: "5216edb8c69da3a25c83c8a0736d6ad9"
server_key_version: "encryptionkey@0"
I20260806 20:25:14.865825 21609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 20:25:14.869894 21624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:14.871511 21609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:25:14.871786 21609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "3368842aa71e46c3b4396f5f330c1361"
format_stamp: "Formatted at 2026-08-06 20:25:14 on dist-test-slave-6k22"
server_key: "75bd941a2efee36668efc711c5afea05"
server_key_iv: "5216edb8c69da3a25c83c8a0736d6ad9"
server_key_version: "encryptionkey@0"
I20260806 20:25:14.871981 21609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:14.893795 21609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:14.898121 21609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:14.898409 21609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:14.917968 21609 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:34525
I20260806 20:25:14.918015 21676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:34525 every 8 connection(s)
I20260806 20:25:14.919399 21609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:14.922561 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 21609
I20260806 20:25:14.922789 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:14.923477 17847 external_mini_cluster.cc:1428] Setting key 5f97be3004d4c94c42c5ed3bef85c02f
I20260806 20:25:14.923513 21677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:14.938942 21677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361: Bootstrap starting.
Aug 06 20:25:14 dist-test-slave-6k22 krb5kdc[21596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:14.942517 21677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:14.943781 21677 log.cc:824] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:14.946731 21677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361: No bootstrap required, opened a new log
I20260806 20:25:14.948956 21680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:14.925489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.17.109.254:34525)
0806 20:25:14.926289 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:14.926301 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:14.926346 (+ 45us) server_negotiation.cc:407] Connection header received
0806 20:25:14.928350 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:14.928373 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:14.928810 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:14.929160 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:14.931057 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:14.932485 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:14.934384 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:14.935330 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:14.935510 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:14.940563 (+ 5053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:14.940574 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:14.940590 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:14.940632 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:14.944425 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:14.945226 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:14.945234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:14.945240 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:14.945435 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:14.945884 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:14.945890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:14.945894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:14.946476 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:14.946802 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:14.947342 (+ 540us) server_negotiation.cc:299] Negotiation successful
0806 20:25:14.947862 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":371,"threads_started":1}
I20260806 20:25:14.955116 21677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3368842aa71e46c3b4396f5f330c1361" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34525 } }
I20260806 20:25:14.955497 21677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:14.955626 21677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3368842aa71e46c3b4396f5f330c1361, State: Initialized, Role: FOLLOWER
I20260806 20:25:14.956188 21677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3368842aa71e46c3b4396f5f330c1361" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34525 } }
I20260806 20:25:14.956454 21677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:14.956609 21677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:14.956779 21677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:14.959113 21677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3368842aa71e46c3b4396f5f330c1361" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34525 } }
I20260806 20:25:14.959663 21677 leader_election.cc:302] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3368842aa71e46c3b4396f5f330c1361; no voters:
I20260806 20:25:14.960315 21677 leader_election.cc:288] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:14.960477 21682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:14.961380 21682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [term 1 LEADER]: Becoming Leader. State: Replica: 3368842aa71e46c3b4396f5f330c1361, State: Running, Role: LEADER
I20260806 20:25:14.962045 21682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3368842aa71e46c3b4396f5f330c1361" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34525 } }
I20260806 20:25:14.962687 21677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:14.965512 21684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3368842aa71e46c3b4396f5f330c1361. Latest consensus state: current_term: 1 leader_uuid: "3368842aa71e46c3b4396f5f330c1361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3368842aa71e46c3b4396f5f330c1361" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34525 } } }
I20260806 20:25:14.966017 21684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:14.965534 21683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3368842aa71e46c3b4396f5f330c1361" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3368842aa71e46c3b4396f5f330c1361" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34525 } } }
I20260806 20:25:14.967968 21683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:14.968180 21687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:14.974400 21687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:14.984467 21687 catalog_manager.cc:1347] Generated new cluster ID: 9fa286eac68743b19fc2ff974ad23791
I20260806 20:25:14.984597 21687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:14.994467 21687 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:14.996409 21687 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:15.008461 21687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3368842aa71e46c3b4396f5f330c1361: Generated new TSK 0
I20260806 20:25:15.009670 21687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:25:15.076025 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34525
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:15.406477 21703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:15.406833 21703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:15.406934 21703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:15.414075 21703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:15.414180 21703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:15.414299 21703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:15.424018 21703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:15.425256 21703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:15.426544 21703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:15.439692 21708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:15.439900 21711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:15.439894 21709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:15.441218 21703 server_base.cc:1034] running on GCE node
I20260806 20:25:15.441857 21703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:15.442837 21703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:15.444061 21703 hybrid_clock.cc:648] HybridClock initialized: now 1786047915444005 us; error 59 us; skew 500 ppm
I20260806 20:25:15.444379 21703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:15 dist-test-slave-6k22 krb5kdc[21596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047915, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:15.447829 21703 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:25:15.449376 21703 webserver.cc:460] Webserver started at http://127.17.109.193:42573/ using document root <none> and password file <none>
I20260806 20:25:15.450052 21703 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:15.450191 21703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:15.450528 21703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:15.453507 21703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "83aaa386dd8a4b6ca2c25f7c312581fb"
format_stamp: "Formatted at 2026-08-06 20:25:15 on dist-test-slave-6k22"
server_key: "a21d9bed58a3dd241204a281b0af0830"
server_key_iv: "ee864a09bb40de6e74851694eb6ef27d"
server_key_version: "encryptionkey@0"
I20260806 20:25:15.454234 21703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "83aaa386dd8a4b6ca2c25f7c312581fb"
format_stamp: "Formatted at 2026-08-06 20:25:15 on dist-test-slave-6k22"
server_key: "a21d9bed58a3dd241204a281b0af0830"
server_key_iv: "ee864a09bb40de6e74851694eb6ef27d"
server_key_version: "encryptionkey@0"
I20260806 20:25:15.459501 21703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:25:15.462989 21718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:15.464792 21703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:15.464967 21703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "83aaa386dd8a4b6ca2c25f7c312581fb"
format_stamp: "Formatted at 2026-08-06 20:25:15 on dist-test-slave-6k22"
server_key: "a21d9bed58a3dd241204a281b0af0830"
server_key_iv: "ee864a09bb40de6e74851694eb6ef27d"
server_key_version: "encryptionkey@0"
I20260806 20:25:15.465174 21703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:15.486598 21703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:15.490909 21703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:15.491237 21703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:15.492334 21703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:15.493741 21703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:15.493814 21703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:15.493978 21703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:15.494053 21703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:15.530673 21703 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:36189
I20260806 20:25:15.530786 21839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:36189 every 8 connection(s)
I20260806 20:25:15.532155 21703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:15.539350 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 21703
I20260806 20:25:15.539620 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:15.540347 17847 external_mini_cluster.cc:1428] Setting key 8837b1c77289f70e382e88ab9a85221a
Aug 06 20:25:15 dist-test-slave-6k22 krb5kdc[21596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047915, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:15.556071 21842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:15.535083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:49347 (local address 127.17.109.254:34525)
0806 20:25:15.535638 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:15.535644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:15.536739 (+ 1095us) server_negotiation.cc:407] Connection header received
0806 20:25:15.538780 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:15.538786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:15.538910 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:15.539045 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:15.540904 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:15.541918 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:15.543889 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:15.544394 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:15.544548 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:15.548594 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:15.548599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:15.548602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:15.548636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:15.552358 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:15.553181 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:15.553190 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:15.553198 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:15.553365 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:15.553848 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:15.553856 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:15.553863 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:15.554306 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:15.554473 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:15.555231 (+ 758us) server_negotiation.cc:299] Negotiation successful
0806 20:25:15.555519 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":233,"threads_started":1}
I20260806 20:25:15.556763 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:15.535724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34525 (local address 127.17.109.193:49347)
0806 20:25:15.536494 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:15.536539 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:15.538473 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:15.539321 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:15.539330 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:15.540029 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:15.540707 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:15.540727 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:15.542093 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:15.542106 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:15.543733 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:15.543746 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:15.544667 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:15.544678 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:15.544945 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:15.545715 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:15.545753 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:15.548331 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:15.552541 (+ 4210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:15.552549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:15.552563 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:15.553021 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:15.553500 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:15.553503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:15.553505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:15.553706 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:15.554444 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:15.554450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:15.554797 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 20:25:15.555032 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 20:25:15.555467 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":287,"threads_started":1}
I20260806 20:25:15.558495 21840 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34525
I20260806 20:25:15.558826 21840 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:15.559617 21840 heartbeater.cc:511] Master 127.17.109.254:34525 requested a full tablet report, sending...
I20260806 20:25:15.561844 21641 ts_manager.cc:194] Registered new tserver with Master: 83aaa386dd8a4b6ca2c25f7c312581fb (127.17.109.193:36189)
I20260806 20:25:15.563424 21641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:49347
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:25:15.599362 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34525
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:15.940855 21846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:15.941282 21846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:15.941475 21846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:15.948833 21846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:15.949016 21846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:15.949218 21846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:15.959949 21846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:15.961413 21846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:15.963094 21846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:15.978472 21851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:15.978502 21853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:15.978546 21855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:15.979431 21846 server_base.cc:1034] running on GCE node
I20260806 20:25:15.980165 21846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:15.981240 21846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:15.982447 21846 hybrid_clock.cc:648] HybridClock initialized: now 1786047915982379 us; error 61 us; skew 500 ppm
I20260806 20:25:15.982774 21846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:15 dist-test-slave-6k22 krb5kdc[21596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047915, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:15.986112 21846 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:25:15.987686 21846 webserver.cc:460] Webserver started at http://127.17.109.194:41503/ using document root <none> and password file <none>
I20260806 20:25:15.988407 21846 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:15.988583 21846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:15.988966 21846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:15.992417 21846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "bb2402a22ea94914bc73dcb951784e64"
format_stamp: "Formatted at 2026-08-06 20:25:15 on dist-test-slave-6k22"
server_key: "c355290d33213eefd2b9be7329d900f1"
server_key_iv: "20b26cbf1c6c305459304a208218efb8"
server_key_version: "encryptionkey@0"
I20260806 20:25:15.993219 21846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "bb2402a22ea94914bc73dcb951784e64"
format_stamp: "Formatted at 2026-08-06 20:25:15 on dist-test-slave-6k22"
server_key: "c355290d33213eefd2b9be7329d900f1"
server_key_iv: "20b26cbf1c6c305459304a208218efb8"
server_key_version: "encryptionkey@0"
I20260806 20:25:15.998580 21846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:25:16.002226 21863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:16.003756 21846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 20:25:16.003944 21846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "bb2402a22ea94914bc73dcb951784e64"
format_stamp: "Formatted at 2026-08-06 20:25:15 on dist-test-slave-6k22"
server_key: "c355290d33213eefd2b9be7329d900f1"
server_key_iv: "20b26cbf1c6c305459304a208218efb8"
server_key_version: "encryptionkey@0"
I20260806 20:25:16.004151 21846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:16.016702 21846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:16.021585 21846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:16.021916 21846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:16.023021 21846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:16.024530 21846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:16.024642 21846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:16.024808 21846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:16.024888 21846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:16.062674 21846 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:38103
I20260806 20:25:16.062772 21980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:38103 every 8 connection(s)
I20260806 20:25:16.064296 21846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:16.073850 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 21846
I20260806 20:25:16.074034 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:16.074579 17847 external_mini_cluster.cc:1428] Setting key e97f0327190b14c5f893945903f32adb
Aug 06 20:25:16 dist-test-slave-6k22 krb5kdc[21596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047915, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:16.089460 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.067102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:52591 (local address 127.17.109.254:34525)
0806 20:25:16.067660 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:16.067665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:16.068692 (+ 1027us) server_negotiation.cc:407] Connection header received
0806 20:25:16.071186 (+ 2494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:16.071193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:16.071300 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:16.071430 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:16.073210 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.074318 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.076777 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.077368 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.077507 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.081925 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:16.081935 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:16.081943 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:16.081993 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:16.085665 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.086644 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.086650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.086653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.086807 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.087380 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.087384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.087387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.087757 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:16.087880 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:16.088956 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0806 20:25:16.089154 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":292,"threads_started":1}
I20260806 20:25:16.091097 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.067626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34525 (local address 127.17.109.194:52591)
0806 20:25:16.068454 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:16.068564 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:16.070816 (+ 2252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:16.071790 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:16.071800 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:16.072362 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:16.073021 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:16.073040 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.074484 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.074489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:16.076575 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:16.076607 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.077605 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.077610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:16.077844 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.078635 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:16.078660 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:16.081620 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:16.085836 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:16.085847 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:16.085867 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:16.086487 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:16.086940 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:16.086948 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:16.086956 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:16.087232 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:16.087889 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:16.087901 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:16.088419 (+ 518us) client_negotiation.cc:769] Sending connection context
0806 20:25:16.088782 (+ 363us) client_negotiation.cc:240] Negotiation successful
0806 20:25:16.089470 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":314,"threads_started":1}
I20260806 20:25:16.092875 21981 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34525
I20260806 20:25:16.093255 21981 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:16.094110 21981 heartbeater.cc:511] Master 127.17.109.254:34525 requested a full tablet report, sending...
I20260806 20:25:16.096005 21641 ts_manager.cc:194] Registered new tserver with Master: bb2402a22ea94914bc73dcb951784e64 (127.17.109.194:38103)
I20260806 20:25:16.096963 21641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:52591
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:25:16.130751 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34525
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:39517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:16.460937 21987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:16.461263 21987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:16.461433 21987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:16.469074 21987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:16.469254 21987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:16.469470 21987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:16.479694 21987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:39517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34525
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:16.481112 21987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:16.482553 21987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:16.496376 21992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:16.496338 21995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:16.496428 21993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:16.497507 21987 server_base.cc:1034] running on GCE node
I20260806 20:25:16.498243 21987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:16.499318 21987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:16.500523 21987 hybrid_clock.cc:648] HybridClock initialized: now 1786047916500434 us; error 82 us; skew 500 ppm
I20260806 20:25:16.500856 21987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:16 dist-test-slave-6k22 krb5kdc[21596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047916, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:16.504384 21987 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:25:16.505853 21987 webserver.cc:460] Webserver started at http://127.17.109.195:41581/ using document root <none> and password file <none>
I20260806 20:25:16.506502 21987 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:16.506600 21987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:16.506857 21987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:16.510141 21987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "e862515c14674c93a4f9bce62243ed8c"
format_stamp: "Formatted at 2026-08-06 20:25:16 on dist-test-slave-6k22"
server_key: "9d993de9f9f107093358bc45a2a58e31"
server_key_iv: "7e009f78c76e46ed344959ebd019f492"
server_key_version: "encryptionkey@0"
I20260806 20:25:16.511150 21987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "e862515c14674c93a4f9bce62243ed8c"
format_stamp: "Formatted at 2026-08-06 20:25:16 on dist-test-slave-6k22"
server_key: "9d993de9f9f107093358bc45a2a58e31"
server_key_iv: "7e009f78c76e46ed344959ebd019f492"
server_key_version: "encryptionkey@0"
I20260806 20:25:16.516623 21987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 20:25:16.520496 22002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:16.521903 21987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:16.522095 21987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "e862515c14674c93a4f9bce62243ed8c"
format_stamp: "Formatted at 2026-08-06 20:25:16 on dist-test-slave-6k22"
server_key: "9d993de9f9f107093358bc45a2a58e31"
server_key_iv: "7e009f78c76e46ed344959ebd019f492"
server_key_version: "encryptionkey@0"
I20260806 20:25:16.522347 21987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:16.554958 21987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:16.559921 21987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:16.560268 21987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:16.561326 21987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:16.562820 21987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:16.562932 21987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:16.563122 21987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:16.563210 21987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:16.566560 21840 heartbeater.cc:503] Master 127.17.109.254:34525 was elected leader, sending a full tablet report...
I20260806 20:25:16.601792 21987 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:37577
I20260806 20:25:16.601879 22118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:37577 every 8 connection(s)
I20260806 20:25:16.603209 21987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:16.603801 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 21987
I20260806 20:25:16.604020 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:16.604662 17847 external_mini_cluster.cc:1428] Setting key b7b317c3d3db2d231972966f888fa41b
Aug 06 20:25:16 dist-test-slave-6k22 krb5kdc[21596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047916, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:16.625679 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.606114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:54903 (local address 127.17.109.254:34525)
0806 20:25:16.606595 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:16.606600 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:16.608163 (+ 1563us) server_negotiation.cc:407] Connection header received
0806 20:25:16.610258 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:16.610262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:16.610332 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:16.610430 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:16.612136 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.613067 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.614778 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.615204 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.615306 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.619255 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:16.619262 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:16.619267 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:16.619329 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:16.622546 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.623344 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.623348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.623350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.623478 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.624042 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.624045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.624048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.624336 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:16.624468 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:16.625150 (+ 682us) server_negotiation.cc:299] Negotiation successful
0806 20:25:16.625325 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":248,"threads_started":1}
I20260806 20:25:16.626667 22122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.606882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34525 (local address 127.17.109.195:54903)
0806 20:25:16.607959 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:16.608025 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:16.609983 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:16.610707 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:16.610716 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:16.611264 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:16.611962 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:16.611981 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.613224 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.613228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:16.614576 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:16.614588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.615398 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.615403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:16.615724 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.616400 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:16.616423 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:16.618985 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:16.622677 (+ 3692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:16.622686 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:16.622703 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:16.623125 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:16.623669 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:16.623673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:16.623676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:16.623894 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:16.624480 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:16.624486 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:16.624807 (+ 321us) client_negotiation.cc:769] Sending connection context
0806 20:25:16.625050 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 20:25:16.625445 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":424,"threads_started":1}
I20260806 20:25:16.628259 22119 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34525
I20260806 20:25:16.628595 22119 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:16.629412 22119 heartbeater.cc:511] Master 127.17.109.254:34525 requested a full tablet report, sending...
I20260806 20:25:16.630913 21641 ts_manager.cc:194] Registered new tserver with Master: e862515c14674c93a4f9bce62243ed8c (127.17.109.195:37577)
I20260806 20:25:16.631984 21641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:54903
I20260806 20:25:16.633752 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 20:25:16 dist-test-slave-6k22 krb5kdc[21596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 20:25:16 dist-test-slave-6k22 krb5kdc[21596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:25:16.693037 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.677136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59272 (local address 127.17.109.193:36189)
0806 20:25:16.677767 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:16.677773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:16.677802 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:25:16.677862 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:16.677869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:16.678081 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:16.678218 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:16.679775 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.680794 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.682075 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.682693 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.682796 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.686830 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:16.686842 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:16.686857 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:16.686892 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:16.689957 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.690749 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.690759 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.690762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.690892 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.691318 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.691323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.691325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.691774 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:16.692028 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:16.692382 (+ 354us) server_negotiation.cc:299] Negotiation successful
0806 20:25:16.692582 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":315,"threads_started":1}
Aug 06 20:25:16 dist-test-slave-6k22 krb5kdc[21596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:16.712301 22121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.697504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46802 (local address 127.17.109.254:34525)
0806 20:25:16.697734 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:16.697741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:16.697890 (+ 149us) server_negotiation.cc:407] Connection header received
0806 20:25:16.698048 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:16.698052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:16.698123 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:16.698216 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:16.699532 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.700377 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.701645 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.702208 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.702308 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.706779 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:16.706784 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:16.706787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:16.706821 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:16.709582 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.710307 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.710311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.710314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.710454 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.710836 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.710840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.710842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.711180 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:16.711376 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:16.711756 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 20:25:16.711943 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260806 20:25:16.712929 21641 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46802
I20260806 20:25:16.714107 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21703
I20260806 20:25:16.727102 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.714985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.17.109.193:36189)
0806 20:25:16.715148 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:16.715153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:16.715434 (+ 281us) server_negotiation.cc:407] Connection header received
0806 20:25:16.715610 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:16.715618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:16.715733 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:16.715815 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:16.717032 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.717737 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.718939 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.719345 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.719447 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.721873 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:16.721878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:16.721881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:16.721913 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:16.724636 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.725263 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.725267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.725270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.725392 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.725782 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.725785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.725787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.726069 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:16.726257 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:16.726582 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 20:25:16.726823 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 20:25:16.727864 21834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:16.810920 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 21703
I20260806 20:25:16.829668 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21846
Aug 06 20:25:16 dist-test-slave-6k22 krb5kdc[21596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:25:16.847343 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.830762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.17.109.194:38103)
0806 20:25:16.831314 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:16.831322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:16.831348 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:25:16.831409 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:16.831415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:16.831768 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:16.831950 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:16.833282 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.834348 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.835723 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.836319 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.836483 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.840843 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:16.840854 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:16.840869 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:16.840905 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:16.843945 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.844678 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.844686 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.844731 (+ 45us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.844906 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.845506 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.845513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.845517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.845942 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:16.846324 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:16.846651 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 20:25:16.846944 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":268,"threads_started":1}
I20260806 20:25:16.848678 21976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:16.932890 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 21846
I20260806 20:25:16.952235 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21987
Aug 06 20:25:16 dist-test-slave-6k22 krb5kdc[21596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:25:16.970606 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:16.953315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.17.109.195:37577)
0806 20:25:16.954093 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:16.954100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:16.954137 (+ 37us) server_negotiation.cc:407] Connection header received
0806 20:25:16.954204 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:16.954211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:16.954467 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:16.954612 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:16.956022 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.957237 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.958574 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:16.959194 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:16.959315 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:16.964506 (+ 5191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:16.964518 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:16.964534 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:16.964570 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:16.967704 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.968439 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.968453 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.968455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.968580 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:16.968976 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:16.968981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:16.968984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:16.969333 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:16.969656 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:16.969959 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 20:25:16.970248 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":447,"threads_started":1}
I20260806 20:25:16.972182 22114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:17.057834 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 21987
I20260806 20:25:17.078840 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 21609
I20260806 20:25:17.080225 21672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:17.185170 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 21609
2026-08-06T20:25:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4149 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:17 dist-test-slave-6k22 krb5kdc[22139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:17 dist-test-slave-6k22 krb5kdc[22139](info): set up 2 sockets
Aug 06 20:25:17 dist-test-slave-6k22 krb5kdc[22139](info): commencing operation
krb5kdc: starting...
W20260806 20:25:18.430368 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:18 dist-test-slave-6k22 krb5kdc[22139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:25:18.671949 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:40067
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:40067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:19.017210 22152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:19.017506 22152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:19.017661 22152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:19.025241 22152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:19.025362 22152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:19.025442 22152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:19.025550 22152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:19.025637 22152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:19.036300 22152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:40067
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:40067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:19.037714 22152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:19.039153 22152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:19.051499 22159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:19.051499 22160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:19.051774 22162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:19.052497 22152 server_base.cc:1034] running on GCE node
I20260806 20:25:19.053367 22152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:19.054912 22152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:19.056157 22152 hybrid_clock.cc:648] HybridClock initialized: now 1786047919056108 us; error 68 us; skew 500 ppm
I20260806 20:25:19.056541 22152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:19 dist-test-slave-6k22 krb5kdc[22139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047919, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:19.068459 22152 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:25:19.069943 22152 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:19.070062 22152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:19.070400 22152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:19.074180 22152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "11dae2af69a64ab6b2f522a58ab74352"
format_stamp: "Formatted at 2026-08-06 20:25:19 on dist-test-slave-6k22"
server_key: "974e70b1dfa48b713b4daf10a8d9fbe1"
server_key_iv: "fac369dccb9de91cf3814826f07af1eb"
server_key_version: "encryptionkey@0"
I20260806 20:25:19.075150 22152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "11dae2af69a64ab6b2f522a58ab74352"
format_stamp: "Formatted at 2026-08-06 20:25:19 on dist-test-slave-6k22"
server_key: "974e70b1dfa48b713b4daf10a8d9fbe1"
server_key_iv: "fac369dccb9de91cf3814826f07af1eb"
server_key_version: "encryptionkey@0"
I20260806 20:25:19.081857 22152 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260806 20:25:19.086491 22168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:19.088233 22152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 20:25:19.088449 22152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "11dae2af69a64ab6b2f522a58ab74352"
format_stamp: "Formatted at 2026-08-06 20:25:19 on dist-test-slave-6k22"
server_key: "974e70b1dfa48b713b4daf10a8d9fbe1"
server_key_iv: "fac369dccb9de91cf3814826f07af1eb"
server_key_version: "encryptionkey@0"
I20260806 20:25:19.088696 22152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:19.105480 22152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:19.109969 22152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:19.110270 22152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:19.129715 22152 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:40067
I20260806 20:25:19.129736 22220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:40067 every 8 connection(s)
I20260806 20:25:19.130673 22152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:19.134418 22221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:19.137997 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 22152
I20260806 20:25:19.138212 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:19.139066 17847 external_mini_cluster.cc:1428] Setting key bd645a9bf58ea15b1167853a82f3d1cb
I20260806 20:25:19.148043 22221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352: Bootstrap starting.
I20260806 20:25:19.152009 22221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:19.153388 22221 log.cc:824] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352: Log is configured to *not* fsync() on all Append() calls
Aug 06 20:25:19 dist-test-slave-6k22 krb5kdc[22139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:19.156750 22221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352: No bootstrap required, opened a new log
I20260806 20:25:19.163025 22224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:19.141196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.17.109.254:40067)
0806 20:25:19.142005 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:19.142016 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:19.142057 (+ 41us) server_negotiation.cc:407] Connection header received
0806 20:25:19.143773 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:19.143799 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:19.144202 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:19.144553 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:19.146458 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:19.147822 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:19.149557 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:19.150241 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:19.150410 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:19.155543 (+ 5133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:19.155552 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:19.155623 (+ 71us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:19.155658 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:19.158775 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:19.159658 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:19.159668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:19.159674 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:19.159847 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:19.160270 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:19.160274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:19.160282 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:19.160843 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:19.161043 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:19.161548 (+ 505us) server_negotiation.cc:299] Negotiation successful
0806 20:25:19.161884 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":384,"threads_started":1}
I20260806 20:25:19.166154 22221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dae2af69a64ab6b2f522a58ab74352" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40067 } }
I20260806 20:25:19.166533 22221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:19.166640 22221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11dae2af69a64ab6b2f522a58ab74352, State: Initialized, Role: FOLLOWER
I20260806 20:25:19.167191 22221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dae2af69a64ab6b2f522a58ab74352" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40067 } }
I20260806 20:25:19.167447 22221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:19.167650 22221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:19.167837 22221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:19.170310 22221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dae2af69a64ab6b2f522a58ab74352" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40067 } }
I20260806 20:25:19.170800 22221 leader_election.cc:302] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11dae2af69a64ab6b2f522a58ab74352; no voters:
I20260806 20:25:19.171406 22221 leader_election.cc:288] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:19.171590 22226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:19.172309 22226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [term 1 LEADER]: Becoming Leader. State: Replica: 11dae2af69a64ab6b2f522a58ab74352, State: Running, Role: LEADER
I20260806 20:25:19.172905 22226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dae2af69a64ab6b2f522a58ab74352" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40067 } }
I20260806 20:25:19.173660 22221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:19.175812 22228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11dae2af69a64ab6b2f522a58ab74352. Latest consensus state: current_term: 1 leader_uuid: "11dae2af69a64ab6b2f522a58ab74352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dae2af69a64ab6b2f522a58ab74352" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40067 } } }
I20260806 20:25:19.176250 22228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:19.177774 22227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11dae2af69a64ab6b2f522a58ab74352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11dae2af69a64ab6b2f522a58ab74352" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40067 } } }
I20260806 20:25:19.178911 22233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:19.179128 22227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:19.184679 22233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:19.194495 22233 catalog_manager.cc:1347] Generated new cluster ID: 267ef316737b49c18aa70804f45dda3c
I20260806 20:25:19.194625 22233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:19.240929 22233 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:19.242304 22233 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:19.263922 22233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352: Generated new TSK 0
I20260806 20:25:19.264838 22233 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:25:19.341754 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40067
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:19.695911 22249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:19.696228 22249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:19.696343 22249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:19.704000 22249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:19.704146 22249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:19.704288 22249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:19.714612 22249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:19.715991 22249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:19.717339 22249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:19.731494 22256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:19.731580 22255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:19.731945 22258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:19.732504 22249 server_base.cc:1034] running on GCE node
I20260806 20:25:19.733165 22249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:19.734174 22249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:19.735414 22249 hybrid_clock.cc:648] HybridClock initialized: now 1786047919735364 us; error 43 us; skew 500 ppm
I20260806 20:25:19.735808 22249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:19 dist-test-slave-6k22 krb5kdc[22139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047919, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:19.739024 22249 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:25:19.740388 22249 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:19.740523 22249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:19.740861 22249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:19.743963 22249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "40870f8ac1e64104a7d42fd8799b9b8f"
format_stamp: "Formatted at 2026-08-06 20:25:19 on dist-test-slave-6k22"
server_key: "37bf00b1fc0331d0a2e5bdb52fcfb750"
server_key_iv: "dc3f9e40ecdd555a1fab6e83215fe210"
server_key_version: "encryptionkey@0"
I20260806 20:25:19.744819 22249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "40870f8ac1e64104a7d42fd8799b9b8f"
format_stamp: "Formatted at 2026-08-06 20:25:19 on dist-test-slave-6k22"
server_key: "37bf00b1fc0331d0a2e5bdb52fcfb750"
server_key_iv: "dc3f9e40ecdd555a1fab6e83215fe210"
server_key_version: "encryptionkey@0"
I20260806 20:25:19.750270 22249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 20:25:19.754015 22265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:19.755401 22249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 20:25:19.755646 22249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "40870f8ac1e64104a7d42fd8799b9b8f"
format_stamp: "Formatted at 2026-08-06 20:25:19 on dist-test-slave-6k22"
server_key: "37bf00b1fc0331d0a2e5bdb52fcfb750"
server_key_iv: "dc3f9e40ecdd555a1fab6e83215fe210"
server_key_version: "encryptionkey@0"
I20260806 20:25:19.755889 22249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:19.773434 22249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:19.777755 22249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:19.778048 22249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:19.779066 22249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:19.780596 22249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:19.780715 22249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:19.780859 22249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:19.780956 22249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:19.816473 22249 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:39703
I20260806 20:25:19.816519 22379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:39703 every 8 connection(s)
I20260806 20:25:19.817374 22249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:19.820540 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 22249
I20260806 20:25:19.820806 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:19.821969 17847 external_mini_cluster.cc:1428] Setting key 1d952a9bd6291bfa88cf979f05e59d7a
Aug 06 20:25:19 dist-test-slave-6k22 krb5kdc[22139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047919, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:19.838840 22382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:19.820513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:49751 (local address 127.17.109.254:40067)
0806 20:25:19.822689 (+ 2176us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:19.822696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:19.822721 (+ 25us) server_negotiation.cc:407] Connection header received
0806 20:25:19.824392 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:19.824399 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:19.824494 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:19.824611 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:19.826111 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:19.826838 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:19.828501 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:19.828902 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:19.829039 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:19.832812 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:19.832817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:19.832820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:19.832852 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:19.835371 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:19.836389 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:19.836393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:19.836396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:19.836519 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:19.837007 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:19.837011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:19.837013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:19.837297 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:19.837412 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:19.838358 (+ 946us) server_negotiation.cc:299] Negotiation successful
0806 20:25:19.838544 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2039,"thread_start_us":1929,"threads_started":1}
I20260806 20:25:19.840423 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:19.821050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40067 (local address 127.17.109.193:49751)
0806 20:25:19.821888 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:19.821950 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:19.824104 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:19.824851 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:19.824859 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:19.825372 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:19.825939 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:19.825960 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:19.827002 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:19.827007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:19.828342 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:19.828354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:19.829128 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:19.829135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:19.829357 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:19.830043 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:19.830065 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:19.832559 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:19.835521 (+ 2962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:19.835533 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:19.835553 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:19.836245 (+ 692us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:19.836627 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:19.836634 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:19.836641 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:19.836887 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:19.837434 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:19.837445 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:19.837935 (+ 490us) client_negotiation.cc:769] Sending connection context
0806 20:25:19.838243 (+ 308us) client_negotiation.cc:240] Negotiation successful
0806 20:25:19.838857 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":369,"threads_started":1}
I20260806 20:25:19.842163 22380 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40067
I20260806 20:25:19.842514 22380 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:19.843426 22380 heartbeater.cc:511] Master 127.17.109.254:40067 requested a full tablet report, sending...
I20260806 20:25:19.845634 22185 ts_manager.cc:194] Registered new tserver with Master: 40870f8ac1e64104a7d42fd8799b9b8f (127.17.109.193:39703)
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
I20260806 20:25:19.847355 22185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:49751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:25:19.878921 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40067
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:20.221900 22387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:20.222221 22387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:20.222407 22387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:20.229749 22387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:20.229899 22387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:20.230085 22387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:20.239952 22387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:20.241277 22387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:20.242602 22387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:20.256428 22393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:20.256376 22392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:20.256467 22396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:20.257121 22387 server_base.cc:1034] running on GCE node
I20260806 20:25:20.257786 22387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:20.258798 22387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:20.260066 22387 hybrid_clock.cc:648] HybridClock initialized: now 1786047920260019 us; error 56 us; skew 500 ppm
I20260806 20:25:20.260483 22387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:20 dist-test-slave-6k22 krb5kdc[22139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047920, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:20.264096 22387 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:25:20.265308 22387 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:20.265451 22387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:20.265807 22387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:20.269315 22387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "b868d48a9eb448bc86cfbd35724cd967"
format_stamp: "Formatted at 2026-08-06 20:25:20 on dist-test-slave-6k22"
server_key: "3ef6e29b5bc4807b6e8608ba8165cec8"
server_key_iv: "4c4560a824420aa35aac3db92e918ae0"
server_key_version: "encryptionkey@0"
I20260806 20:25:20.270289 22387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "b868d48a9eb448bc86cfbd35724cd967"
format_stamp: "Formatted at 2026-08-06 20:25:20 on dist-test-slave-6k22"
server_key: "3ef6e29b5bc4807b6e8608ba8165cec8"
server_key_iv: "4c4560a824420aa35aac3db92e918ae0"
server_key_version: "encryptionkey@0"
I20260806 20:25:20.275887 22387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260806 20:25:20.279511 22402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:20.281145 22387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:20.281327 22387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "b868d48a9eb448bc86cfbd35724cd967"
format_stamp: "Formatted at 2026-08-06 20:25:20 on dist-test-slave-6k22"
server_key: "3ef6e29b5bc4807b6e8608ba8165cec8"
server_key_iv: "4c4560a824420aa35aac3db92e918ae0"
server_key_version: "encryptionkey@0"
I20260806 20:25:20.281556 22387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:20.306917 22387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:20.311772 22387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:20.312088 22387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:20.313194 22387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:20.314721 22387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:20.314880 22387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:20.315019 22387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:20.315121 22387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:20.354303 22387 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:46255
I20260806 20:25:20.354357 22518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:46255 every 8 connection(s)
I20260806 20:25:20.355331 22387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:20.362983 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 22387
I20260806 20:25:20.363173 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:20.363801 17847 external_mini_cluster.cc:1428] Setting key 14dcc8b171eeaa5144ac2290ab4fe4e2
Aug 06 20:25:20 dist-test-slave-6k22 krb5kdc[22139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047920, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:20.380431 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.358105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:39717 (local address 127.17.109.254:40067)
0806 20:25:20.358587 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:20.358592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:20.359620 (+ 1028us) server_negotiation.cc:407] Connection header received
0806 20:25:20.361576 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:20.361583 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:20.361675 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:20.361805 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:20.363790 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.364866 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.367426 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.368212 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.368379 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.371937 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:20.371947 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:20.371955 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:20.372006 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:20.376483 (+ 4477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:20.377569 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:20.377574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:20.377577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:20.377796 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:20.378331 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:20.378335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:20.378337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:20.378666 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:20.378846 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:20.379844 (+ 998us) server_negotiation.cc:299] Negotiation successful
0806 20:25:20.380059 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":274,"threads_started":1}
I20260806 20:25:20.381968 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.358607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40067 (local address 127.17.109.194:39717)
0806 20:25:20.359383 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:20.359422 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:20.361293 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:20.362043 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:20.362051 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:20.362711 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:20.363543 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.363600 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.365033 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.365038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.367256 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.367268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.368429 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.368434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.368619 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.369391 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:20.369415 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:20.371679 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:20.376647 (+ 4968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:20.376659 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:20.376680 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:20.377408 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:20.377923 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:20.377931 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:20.377937 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:20.378202 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:20.378854 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:20.378865 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:20.379347 (+ 482us) client_negotiation.cc:769] Sending connection context
0806 20:25:20.379713 (+ 366us) client_negotiation.cc:240] Negotiation successful
0806 20:25:20.380370 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":308,"threads_started":1}
I20260806 20:25:20.383641 22519 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40067
I20260806 20:25:20.383997 22519 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:20.384711 22519 heartbeater.cc:511] Master 127.17.109.254:40067 requested a full tablet report, sending...
I20260806 20:25:20.386354 22185 ts_manager.cc:194] Registered new tserver with Master: b868d48a9eb448bc86cfbd35724cd967 (127.17.109.194:46255)
I20260806 20:25:20.387290 22185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:39717
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:25:20.423049 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40067
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:20.752418 22525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:20.752750 22525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:20.752923 22525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:20.760905 22525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:20.761049 22525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:20.761240 22525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:20.771097 22525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:20.772568 22525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:20.774000 22525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:20.787711 22530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:20.787756 22533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:20.787972 22531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:20.788957 22525 server_base.cc:1034] running on GCE node
I20260806 20:25:20.789577 22525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:20.790545 22525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:20.791774 22525 hybrid_clock.cc:648] HybridClock initialized: now 1786047920791713 us; error 71 us; skew 500 ppm
I20260806 20:25:20.792137 22525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:20 dist-test-slave-6k22 krb5kdc[22139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047920, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:20.795419 22525 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:25:20.796680 22525 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:20.796809 22525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:20.797119 22525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:20.800076 22525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "889ffe2f76d040ebbd763099c7071d57"
format_stamp: "Formatted at 2026-08-06 20:25:20 on dist-test-slave-6k22"
server_key: "7ebf2fb2eb42296418763d41a5135d98"
server_key_iv: "848bf27aa4b9d75233d652a4c7260808"
server_key_version: "encryptionkey@0"
I20260806 20:25:20.800876 22525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "889ffe2f76d040ebbd763099c7071d57"
format_stamp: "Formatted at 2026-08-06 20:25:20 on dist-test-slave-6k22"
server_key: "7ebf2fb2eb42296418763d41a5135d98"
server_key_iv: "848bf27aa4b9d75233d652a4c7260808"
server_key_version: "encryptionkey@0"
I20260806 20:25:20.805876 22525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:25:20.809257 22539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:20.810722 22525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:25:20.810881 22525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "889ffe2f76d040ebbd763099c7071d57"
format_stamp: "Formatted at 2026-08-06 20:25:20 on dist-test-slave-6k22"
server_key: "7ebf2fb2eb42296418763d41a5135d98"
server_key_iv: "848bf27aa4b9d75233d652a4c7260808"
server_key_version: "encryptionkey@0"
I20260806 20:25:20.811098 22525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:20.844492 22525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:20.848747 22525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:20.849033 22525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:20.850003 22525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:20.850613 22380 heartbeater.cc:503] Master 127.17.109.254:40067 was elected leader, sending a full tablet report...
I20260806 20:25:20.851529 22525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:20.851706 22525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260806 20:25:20.851833 22525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:20.851924 22525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:20.886332 22525 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:36261
I20260806 20:25:20.886420 22653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:36261 every 8 connection(s)
I20260806 20:25:20.887225 22525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:20.897145 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 22525
I20260806 20:25:20.897346 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:20.897797 17847 external_mini_cluster.cc:1428] Setting key 54950598c168034e325c176b8f3977b2
Aug 06 20:25:20 dist-test-slave-6k22 krb5kdc[22139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047920, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:20.906694 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.889742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:42439 (local address 127.17.109.254:40067)
0806 20:25:20.890206 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:20.890212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:20.891247 (+ 1035us) server_negotiation.cc:407] Connection header received
0806 20:25:20.893137 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:20.893142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:20.893263 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:20.893379 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:20.894777 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.895542 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.897108 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.897667 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.897799 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.901622 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:20.901627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:20.901630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:20.901665 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:20.904140 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:20.904819 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:20.904823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:20.904825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:20.904938 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:20.905312 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:20.905315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:20.905317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:20.905556 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:20.905671 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:20.906273 (+ 602us) server_negotiation.cc:299] Negotiation successful
0806 20:25:20.906428 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":207,"threads_started":1}
I20260806 20:25:20.907717 22657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.890355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40067 (local address 127.17.109.195:42439)
0806 20:25:20.891078 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:20.891128 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:20.892892 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:20.893551 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:20.893560 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:20.894052 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:20.894592 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.894609 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.895750 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.895753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.896959 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.896971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.897903 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.897910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.898126 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.898840 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:20.898864 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:20.901444 (+ 2580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:20.904285 (+ 2841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:20.904292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:20.904303 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:20.904714 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:20.905030 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:20.905033 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:20.905035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:20.905222 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:20.905655 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:20.905660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:20.905950 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 20:25:20.906180 (+ 230us) client_negotiation.cc:240] Negotiation successful
0806 20:25:20.906557 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":319,"threads_started":1}
I20260806 20:25:20.909086 22654 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40067
I20260806 20:25:20.909394 22654 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:20.910012 22654 heartbeater.cc:511] Master 127.17.109.254:40067 requested a full tablet report, sending...
I20260806 20:25:20.911466 22185 ts_manager.cc:194] Registered new tserver with Master: 889ffe2f76d040ebbd763099c7071d57 (127.17.109.195:36261)
I20260806 20:25:20.912539 22185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:42439
I20260806 20:25:20.913550 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:25:20.930585 22656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.918304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.17.109.254:40067)
0806 20:25:20.918486 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:20.918490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:20.919054 (+ 564us) server_negotiation.cc:407] Connection header received
0806 20:25:20.919187 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:20.919191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:20.919263 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:20.919344 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:20.920867 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.921590 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.922845 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.923274 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.923374 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.925680 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:20.925686 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:20.925690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:20.925750 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:20.928351 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:20.928937 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:20.928942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:20.928944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:20.929059 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:20.929383 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:20.929386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:20.929388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:20.929720 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:20.929890 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:20.930176 (+ 286us) server_negotiation.cc:299] Negotiation successful
0806 20:25:20.930341 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 20:25:20.934864 22184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:25:20.942222 22184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:20.976259 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.955244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.17.109.195:36261)
0806 20:25:20.966409 (+ 11165us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:20.966417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:20.966444 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:25:20.966524 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:20.966530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:20.966711 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:20.967833 (+ 1122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:20.968165 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.969735 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.973243 (+ 3508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.974325 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.974535 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.974735 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:20.975739 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0806 20:25:20.975911 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10976,"thread_start_us":10824,"threads_started":1}
I20260806 20:25:20.976337 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.954999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:36261 (local address 127.0.0.1:60496)
0806 20:25:20.964881 (+ 9882us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:20.964947 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:20.965203 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:20.966947 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:20.966954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:20.966982 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:20.967530 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.967546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.969880 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.969886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.973064 (+ 3178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.973076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.974536 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.974542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.975376 (+ 834us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.975436 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 20:25:20.975589 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 20:25:20.975823 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9545,"mutex_wait_us":87,"thread_start_us":10695,"threads_started":1}
I20260806 20:25:20.976379 22667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.955289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:39703 (local address 127.0.0.1:33494)
0806 20:25:20.965953 (+ 10664us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:20.965976 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:20.966109 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:20.967146 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:20.967150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:20.967187 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:20.967713 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.967724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.969782 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.969791 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.972835 (+ 3044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.972853 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.975098 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.975107 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.975549 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.975634 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 20:25:20.975764 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 20:25:20.975907 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10537,"mutex_wait_us":110,"thread_start_us":9097,"threads_started":1}
I20260806 20:25:20.977139 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.955662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.17.109.193:39703)
0806 20:25:20.964422 (+ 8760us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:20.964434 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:20.966249 (+ 1815us) server_negotiation.cc:407] Connection header received
0806 20:25:20.966365 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:20.966374 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:20.966644 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:20.966851 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:20.967899 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.969606 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.973063 (+ 3457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.974575 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.974796 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.974990 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:20.976623 (+ 1633us) server_negotiation.cc:299] Negotiation successful
0806 20:25:20.976802 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8524,"thread_start_us":8345,"threads_started":1}
I20260806 20:25:20.981014 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.965805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36684 (local address 127.17.109.194:46255)
0806 20:25:20.971184 (+ 5379us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:20.971190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:20.971627 (+ 437us) server_negotiation.cc:407] Connection header received
0806 20:25:20.971841 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:20.971850 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:20.972098 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:20.972309 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:20.973209 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.974790 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.978436 (+ 3646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.979935 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.980130 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.980308 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:20.980549 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 20:25:20.980707 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5225,"thread_start_us":5111,"threads_started":1}
I20260806 20:25:20.981026 22673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:20.966525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:46255 (local address 127.0.0.1:36684)
0806 20:25:20.971432 (+ 4907us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:20.971463 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:20.971639 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:20.972528 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:20.972533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:20.972557 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:20.973011 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.973028 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.974974 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.974979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.977212 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:20.977227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:20.980150 (+ 2923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:20.980157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:20.980322 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:20.980369 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 20:25:20.980496 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 20:25:20.980628 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4705,"thread_start_us":4667,"threads_started":1}
I20260806 20:25:20.981069 22314 tablet_service.cc:1467] Processing CreateTablet for tablet e91389608e4d4558b0c712265d115038 (DEFAULT_TABLE table=test-table [id=6b645e75567e452186bca3d3e22cfd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:20.981069 22587 tablet_service.cc:1467] Processing CreateTablet for tablet e91389608e4d4558b0c712265d115038 (DEFAULT_TABLE table=test-table [id=6b645e75567e452186bca3d3e22cfd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:20.982640 22314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91389608e4d4558b0c712265d115038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:20.982640 22587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91389608e4d4558b0c712265d115038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:20.987751 22450 tablet_service.cc:1467] Processing CreateTablet for tablet e91389608e4d4558b0c712265d115038 (DEFAULT_TABLE table=test-table [id=6b645e75567e452186bca3d3e22cfd53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:20.989153 22450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91389608e4d4558b0c712265d115038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:20.998791 22675 tablet_bootstrap.cc:492] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Bootstrap starting.
I20260806 20:25:21.001700 22676 tablet_bootstrap.cc:492] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Bootstrap starting.
I20260806 20:25:21.002753 22675 tablet_bootstrap.cc:654] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:21.002967 22674 tablet_bootstrap.cc:492] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Bootstrap starting.
I20260806 20:25:21.004208 22675 log.cc:824] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:21.005406 22676 tablet_bootstrap.cc:654] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:21.006390 22676 log.cc:824] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:21.006421 22674 tablet_bootstrap.cc:654] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:21.007648 22675 tablet_bootstrap.cc:492] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: No bootstrap required, opened a new log
I20260806 20:25:21.007655 22674 log.cc:824] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:21.007989 22675 ts_tablet_manager.cc:1397] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260806 20:25:21.009510 22676 tablet_bootstrap.cc:492] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: No bootstrap required, opened a new log
I20260806 20:25:21.009811 22676 ts_tablet_manager.cc:1397] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 20:25:21.011492 22674 tablet_bootstrap.cc:492] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: No bootstrap required, opened a new log
I20260806 20:25:21.011803 22674 ts_tablet_manager.cc:1397] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260806 20:25:21.017795 22676 raft_consensus.cc:348] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.018190 22676 raft_consensus.cc:374] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:21.018299 22676 raft_consensus.cc:729] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b868d48a9eb448bc86cfbd35724cd967, State: Initialized, Role: FOLLOWER
I20260806 20:25:21.018940 22676 consensus_queue.cc:260] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.019042 22675 raft_consensus.cc:348] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.019531 22675 raft_consensus.cc:374] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:21.019716 22675 raft_consensus.cc:729] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40870f8ac1e64104a7d42fd8799b9b8f, State: Initialized, Role: FOLLOWER
I20260806 20:25:21.020159 22519 heartbeater.cc:503] Master 127.17.109.254:40067 was elected leader, sending a full tablet report...
I20260806 20:25:21.020355 22675 consensus_queue.cc:260] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.020870 22676 ts_tablet_manager.cc:1428] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260806 20:25:21.023773 22674 raft_consensus.cc:348] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.024286 22674 raft_consensus.cc:374] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:21.024410 22674 raft_consensus.cc:729] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 889ffe2f76d040ebbd763099c7071d57, State: Initialized, Role: FOLLOWER
I20260806 20:25:21.025058 22675 ts_tablet_manager.cc:1428] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Time spent starting tablet: real 0.017s user 0.008s sys 0.007s
I20260806 20:25:21.024974 22674 consensus_queue.cc:260] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.026736 22654 heartbeater.cc:503] Master 127.17.109.254:40067 was elected leader, sending a full tablet report...
I20260806 20:25:21.027421 22674 ts_tablet_manager.cc:1428] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260806 20:25:21.064393 22680 raft_consensus.cc:482] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:21.064759 22680 raft_consensus.cc:504] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.066571 22680 leader_election.cc:288] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 889ffe2f76d040ebbd763099c7071d57 (127.17.109.195:36261), 40870f8ac1e64104a7d42fd8799b9b8f (127.17.109.193:39703)
W20260806 20:25:21.070236 22381 tablet.cc:2369] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:25:21.075587 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:21.067177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:50701 (local address 127.17.109.193:39703)
0806 20:25:21.067474 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:21.067483 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:21.068163 (+ 680us) server_negotiation.cc:407] Connection header received
0806 20:25:21.068432 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:21.068436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:21.068497 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:21.068766 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:21.069447 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.070395 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.072808 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.074257 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.074488 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:21.074584 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:21.075146 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 20:25:21.075311 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260806 20:25:21.075577 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:21.067098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:53303 (local address 127.17.109.195:36261)
0806 20:25:21.067377 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:21.067387 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:21.068268 (+ 881us) server_negotiation.cc:407] Connection header received
0806 20:25:21.068481 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:21.068485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:21.068546 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:21.068641 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:21.069314 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.070041 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.072761 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.074375 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.074579 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:21.074685 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:21.075146 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 20:25:21.075287 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 20:25:21.076081 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:21.066993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:36261 (local address 127.17.109.194:53303)
0806 20:25:21.068158 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:21.068172 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:21.068334 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:21.068673 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:21.068678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:21.068703 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:21.069111 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:21.069121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.070170 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.070178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:21.072476 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:21.072492 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.074595 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.074600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:21.074886 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:21.074963 (+ 77us) client_negotiation.cc:769] Sending connection context
0806 20:25:21.075062 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 20:25:21.075276 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"mutex_wait_us":167,"thread_start_us":674,"threads_started":1}
I20260806 20:25:21.076253 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:21.066936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:39703 (local address 127.17.109.194:50701)
0806 20:25:21.068028 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:21.068045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:21.068247 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:21.068789 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:21.068793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:21.068835 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:21.069295 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:21.069304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.070570 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.070575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:21.072638 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:21.072656 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.074576 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.074588 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:21.074889 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:21.074960 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 20:25:21.075062 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 20:25:21.075226 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":68,"spinlock_wait_cycles":43264,"thread_start_us":908,"threads_started":1}
I20260806 20:25:21.077209 22334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91389608e4d4558b0c712265d115038" candidate_uuid: "b868d48a9eb448bc86cfbd35724cd967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" is_pre_election: true
I20260806 20:25:21.077210 22608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91389608e4d4558b0c712265d115038" candidate_uuid: "b868d48a9eb448bc86cfbd35724cd967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889ffe2f76d040ebbd763099c7071d57" is_pre_election: true
I20260806 20:25:21.077719 22334 raft_consensus.cc:2454] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b868d48a9eb448bc86cfbd35724cd967 in term 0.
I20260806 20:25:21.077733 22608 raft_consensus.cc:2454] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b868d48a9eb448bc86cfbd35724cd967 in term 0.
I20260806 20:25:21.078436 22403 leader_election.cc:302] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40870f8ac1e64104a7d42fd8799b9b8f, b868d48a9eb448bc86cfbd35724cd967; no voters:
I20260806 20:25:21.078896 22680 raft_consensus.cc:2781] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:21.079056 22680 raft_consensus.cc:482] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:21.079160 22680 raft_consensus.cc:3037] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:21.081627 22680 raft_consensus.cc:504] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.082363 22680 leader_election.cc:288] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [CANDIDATE]: Term 1 election: Requested vote from peers 889ffe2f76d040ebbd763099c7071d57 (127.17.109.195:36261), 40870f8ac1e64104a7d42fd8799b9b8f (127.17.109.193:39703)
I20260806 20:25:21.082890 22334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91389608e4d4558b0c712265d115038" candidate_uuid: "b868d48a9eb448bc86cfbd35724cd967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40870f8ac1e64104a7d42fd8799b9b8f"
I20260806 20:25:21.082948 22608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91389608e4d4558b0c712265d115038" candidate_uuid: "b868d48a9eb448bc86cfbd35724cd967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "889ffe2f76d040ebbd763099c7071d57"
I20260806 20:25:21.083115 22334 raft_consensus.cc:3037] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:21.083244 22608 raft_consensus.cc:3037] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:21.085783 22334 raft_consensus.cc:2454] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b868d48a9eb448bc86cfbd35724cd967 in term 1.
I20260806 20:25:21.086074 22608 raft_consensus.cc:2454] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b868d48a9eb448bc86cfbd35724cd967 in term 1.
I20260806 20:25:21.086253 22403 leader_election.cc:302] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40870f8ac1e64104a7d42fd8799b9b8f, b868d48a9eb448bc86cfbd35724cd967; no voters:
I20260806 20:25:21.086599 22680 raft_consensus.cc:2781] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:21.087090 22680 raft_consensus.cc:686] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 1 LEADER]: Becoming Leader. State: Replica: b868d48a9eb448bc86cfbd35724cd967, State: Running, Role: LEADER
I20260806 20:25:21.087697 22680 consensus_queue.cc:237] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } }
I20260806 20:25:21.092533 22184 catalog_manager.cc:5495] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 reported cstate change: term changed from 0 to 1, leader changed from <none> to b868d48a9eb448bc86cfbd35724cd967 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "b868d48a9eb448bc86cfbd35724cd967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b868d48a9eb448bc86cfbd35724cd967" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46255 } health_report { overall_health: HEALTHY } } }
W20260806 20:25:21.107390 22520 tablet.cc:2369] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:25:21.129146 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:21.123881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.17.109.194:46255)
0806 20:25:21.124090 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:21.124095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:21.124110 (+ 15us) server_negotiation.cc:407] Connection header received
0806 20:25:21.124205 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:21.124209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:21.124291 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:21.124454 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:21.124842 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.125543 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.126879 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.127307 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.127444 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:21.127994 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:21.128506 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:21.128666 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:21.128726 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 20:25:21.128829 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:25:21.137645 22608 raft_consensus.cc:1261] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 1 FOLLOWER]: Refusing update from remote peer b868d48a9eb448bc86cfbd35724cd967: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:21.137871 22334 raft_consensus.cc:1261] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 1 FOLLOWER]: Refusing update from remote peer b868d48a9eb448bc86cfbd35724cd967: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 20:25:21.138814 22655 tablet.cc:2369] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:25:21.138746 22685 consensus_queue.cc:1035] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "889ffe2f76d040ebbd763099c7071d57" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:21.139550 22680 consensus_queue.cc:1035] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40870f8ac1e64104a7d42fd8799b9b8f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:21.151322 22688 mvcc.cc:205] Tried to move back new op lower bound from 7315652284969529344 to 7315652284779384832. Current Snapshot: MvccSnapshot[applied={T|T < 7315652284969529344}]
I20260806 20:25:21.171459 22184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:21.171759 22184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:21.175984 22184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11dae2af69a64ab6b2f522a58ab74352: Sending DeleteTablet for 3 replicas of tablet e91389608e4d4558b0c712265d115038
I20260806 20:25:21.177273 22587 tablet_service.cc:1514] Processing DeleteTablet for tablet e91389608e4d4558b0c712265d115038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:21 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:60496
I20260806 20:25:21.177330 22314 tablet_service.cc:1514] Processing DeleteTablet for tablet e91389608e4d4558b0c712265d115038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:21 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:33494
I20260806 20:25:21.177590 22450 tablet_service.cc:1514] Processing DeleteTablet for tablet e91389608e4d4558b0c712265d115038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:21 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:36684
I20260806 20:25:21.179490 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 22249
I20260806 20:25:21.190250 22698 tablet_replica.cc:330] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: stopping tablet replica
I20260806 20:25:21.190253 22700 tablet_replica.cc:330] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: stopping tablet replica
I20260806 20:25:21.190773 22698 raft_consensus.cc:2229] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:21.190888 22700 raft_consensus.cc:2229] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:21.191383 22698 raft_consensus.cc:2258] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:21.192798 22699 tablet_replica.cc:330] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: stopping tablet replica
I20260806 20:25:21.193111 22700 raft_consensus.cc:2258] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:21.193236 22699 raft_consensus.cc:2229] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:21.194141 22699 raft_consensus.cc:2258] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:21.194943 22698 ts_tablet_manager.cc:1905] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:21.195789 22700 ts_tablet_manager.cc:1905] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:21.200666 22699 ts_tablet_manager.cc:1905] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:21.203011 22698 ts_tablet_manager.cc:1918] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:21.203258 22698 log.cc:1196] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/e91389608e4d4558b0c712265d115038
I20260806 20:25:21.203780 22700 ts_tablet_manager.cc:1918] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:21.203965 22700 log.cc:1196] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/e91389608e4d4558b0c712265d115038
I20260806 20:25:21.204185 22698 ts_tablet_manager.cc:1939] T e91389608e4d4558b0c712265d115038 P 889ffe2f76d040ebbd763099c7071d57: Deleting consensus metadata
I20260806 20:25:21.204664 22700 ts_tablet_manager.cc:1939] T e91389608e4d4558b0c712265d115038 P b868d48a9eb448bc86cfbd35724cd967: Deleting consensus metadata
I20260806 20:25:21.206425 22171 catalog_manager.cc:4841] TS 889ffe2f76d040ebbd763099c7071d57 (127.17.109.195:36261): tablet e91389608e4d4558b0c712265d115038 (table test-table [id=6b645e75567e452186bca3d3e22cfd53]) successfully deleted
I20260806 20:25:21.206831 22169 catalog_manager.cc:4841] TS b868d48a9eb448bc86cfbd35724cd967 (127.17.109.194:46255): tablet e91389608e4d4558b0c712265d115038 (table test-table [id=6b645e75567e452186bca3d3e22cfd53]) successfully deleted
Aug 06 20:25:21 dist-test-slave-6k22 krb5kdc[22139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:25:21.212376 22699 ts_tablet_manager.cc:1918] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:21.212585 22699 log.cc:1196] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/e91389608e4d4558b0c712265d115038
I20260806 20:25:21.213289 22699 ts_tablet_manager.cc:1939] T e91389608e4d4558b0c712265d115038 P 40870f8ac1e64104a7d42fd8799b9b8f: Deleting consensus metadata
I20260806 20:25:21.215520 22169 catalog_manager.cc:4841] TS 40870f8ac1e64104a7d42fd8799b9b8f (127.17.109.193:39703): tablet e91389608e4d4558b0c712265d115038 (table test-table [id=6b645e75567e452186bca3d3e22cfd53]) successfully deleted
I20260806 20:25:21.215868 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:21.181357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.17.109.193:39703)
0806 20:25:21.181580 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:21.181589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:21.189861 (+ 8272us) server_negotiation.cc:407] Connection header received
0806 20:25:21.190262 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:21.190268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:21.190354 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:21.190429 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:21.192314 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.193797 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.196589 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.197348 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.197470 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:21.208494 (+ 11024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:21.208507 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:21.208522 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:21.208559 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:21.212326 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:21.213211 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:21.213221 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:21.213223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:21.213459 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:21.213891 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:21.213897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:21.213905 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:21.214385 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:21.214758 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:21.215111 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 20:25:21.215367 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:25:21.217036 22375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:21.311225 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 22249
I20260806 20:25:21.339052 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 22387
Aug 06 20:25:21 dist-test-slave-6k22 krb5kdc[22139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:25:21.357318 22672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:21.340024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.17.109.194:46255)
0806 20:25:21.340223 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:21.340228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:21.340247 (+ 19us) server_negotiation.cc:407] Connection header received
0806 20:25:21.340485 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:21.340490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:21.340559 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:21.340660 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:21.342303 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.343214 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.345094 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.345572 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.345696 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:21.350881 (+ 5185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:21.350898 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:21.350914 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:21.350954 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:21.354248 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:21.354967 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:21.354975 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:21.354977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:21.355134 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:21.355532 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:21.355538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:21.355541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:21.355963 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:21.356370 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:21.356574 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 20:25:21.356826 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 20:25:21.358072 22513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:21.472666 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 22387
I20260806 20:25:21.495409 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 22525
Aug 06 20:25:21 dist-test-slave-6k22 krb5kdc[22139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:25:21.511065 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:21.496295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.17.109.195:36261)
0806 20:25:21.496491 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:21.496498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:21.496583 (+ 85us) server_negotiation.cc:407] Connection header received
0806 20:25:21.496849 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:21.496854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:21.496927 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:21.497036 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:21.498302 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.499085 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.500525 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:21.500934 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:21.501071 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:21.505456 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:21.505470 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:21.505487 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:21.505519 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:21.508293 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:21.508947 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:21.508955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:21.508957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:21.509095 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:21.509475 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:21.509480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:21.509483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:21.509852 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:21.510155 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:21.510501 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 20:25:21.510744 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 20:25:21.511951 22649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:21.602375 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 22525
I20260806 20:25:21.621812 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 22152
I20260806 20:25:21.623133 22216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:21.741474 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 22152
2026-08-06T20:25:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4557 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T20:25:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:21Z Disabled control of system clock
I20260806 20:25:21.820894 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:33685
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:33685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 20:25:22.173703 22709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:22.174135 22709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:22.174290 22709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:22.185003 22709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:22.185112 22709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:22.185149 22709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:22.185202 22709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:22.185245 22709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:22.194739 22709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:33685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:33685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:22.196099 22709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:22.197877 22709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:22.209812 22714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:22.209860 22717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:22.209824 22715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:22.211035 22709 server_base.cc:1034] running on GCE node
I20260806 20:25:22.211916 22709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:22.213428 22709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:22.214705 22709 hybrid_clock.cc:648] HybridClock initialized: now 1786047922214588 us; error 105 us; skew 500 ppm
I20260806 20:25:22.215106 22709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:22.217739 22709 webserver.cc:460] Webserver started at http://127.17.109.254:36625/ using document root <none> and password file <none>
I20260806 20:25:22.218515 22709 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:22.218676 22709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:22.219031 22709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:22.222270 22709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "2c669d142c92407aad194851743c5172"
format_stamp: "Formatted at 2026-08-06 20:25:22 on dist-test-slave-6k22"
server_key: "26137f782d95ba9650f5518d714b48b8"
server_key_iv: "5fe58c4bb84fd30b3810984fc99038b4"
server_key_version: "encryptionkey@0"
I20260806 20:25:22.223155 22709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "2c669d142c92407aad194851743c5172"
format_stamp: "Formatted at 2026-08-06 20:25:22 on dist-test-slave-6k22"
server_key: "26137f782d95ba9650f5518d714b48b8"
server_key_iv: "5fe58c4bb84fd30b3810984fc99038b4"
server_key_version: "encryptionkey@0"
I20260806 20:25:22.228874 22709 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 20:25:22.232594 22723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:22.234004 22709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 20:25:22.234203 22709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "2c669d142c92407aad194851743c5172"
format_stamp: "Formatted at 2026-08-06 20:25:22 on dist-test-slave-6k22"
server_key: "26137f782d95ba9650f5518d714b48b8"
server_key_iv: "5fe58c4bb84fd30b3810984fc99038b4"
server_key_version: "encryptionkey@0"
I20260806 20:25:22.234440 22709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:22.240274 22709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:22.241041 22709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:22.241346 22709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:22.260277 22709 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:33685
I20260806 20:25:22.260376 22776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:33685 every 8 connection(s)
I20260806 20:25:22.261650 22709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:22.262919 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 22709
I20260806 20:25:22.263144 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:22.263893 17847 external_mini_cluster.cc:1428] Setting key 0c39555207bf90bc7adf7ba75b616292
I20260806 20:25:22.266346 22778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:22.272590 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:22.265979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44046 (local address 127.17.109.254:33685)
0806 20:25:22.267082 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:22.267092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:22.267132 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:25:22.268799 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:22.268828 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:22.269245 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:22.269683 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:22.270855 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:22.270861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:22.270873 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:22.270888 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:22.270968 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:22.270983 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:22.271236 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:22.271386 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 20:25:22.271691 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":448,"threads_started":1}
I20260806 20:25:22.278756 22778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172: Bootstrap starting.
I20260806 20:25:22.282514 22778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:22.283684 22778 log.cc:824] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:22.286574 22778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172: No bootstrap required, opened a new log
I20260806 20:25:22.295987 22778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c669d142c92407aad194851743c5172" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33685 } }
I20260806 20:25:22.296447 22778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:22.296563 22778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c669d142c92407aad194851743c5172, State: Initialized, Role: FOLLOWER
I20260806 20:25:22.297180 22778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c669d142c92407aad194851743c5172" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33685 } }
I20260806 20:25:22.297410 22778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:22.297516 22778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:22.297663 22778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:22.300323 22778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c669d142c92407aad194851743c5172" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33685 } }
I20260806 20:25:22.300875 22778 leader_election.cc:302] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c669d142c92407aad194851743c5172; no voters:
I20260806 20:25:22.301486 22778 leader_election.cc:288] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:22.301654 22783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:22.302407 22783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [term 1 LEADER]: Becoming Leader. State: Replica: 2c669d142c92407aad194851743c5172, State: Running, Role: LEADER
I20260806 20:25:22.302963 22783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c669d142c92407aad194851743c5172" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33685 } }
I20260806 20:25:22.303821 22778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:22.305698 22785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c669d142c92407aad194851743c5172. Latest consensus state: current_term: 1 leader_uuid: "2c669d142c92407aad194851743c5172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c669d142c92407aad194851743c5172" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33685 } } }
I20260806 20:25:22.306010 22785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:22.306923 22789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:22.306758 22784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c669d142c92407aad194851743c5172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c669d142c92407aad194851743c5172" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33685 } } }
I20260806 20:25:22.308053 22784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:22.312266 22789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:22.322578 22789 catalog_manager.cc:1347] Generated new cluster ID: ceb4fbfbe9e14320a9da685eca4fd010
I20260806 20:25:22.322698 22789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:22.336933 22789 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:22.337455 22789 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:22.349704 22789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172: Generated new TSK 0
I20260806 20:25:22.350943 22789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:25:22.365759 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:33685
--builtin_ntp_servers=127.17.109.212:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 20:25:22.773069 22802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:22.773375 22802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:22.773545 22802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:22.780876 22802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:22.780974 22802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:22.781149 22802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:22.790318 22802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:33685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:22.791638 22802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:22.793416 22802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:22.806389 22807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:22.806969 22808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:22.807283 22810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:22.807945 22802 server_base.cc:1034] running on GCE node
I20260806 20:25:22.808604 22802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:22.810148 22802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:22.811357 22802 hybrid_clock.cc:648] HybridClock initialized: now 1786047922811325 us; error 35 us; skew 500 ppm
I20260806 20:25:22.811728 22802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:22.814033 22802 webserver.cc:460] Webserver started at http://127.17.109.193:46739/ using document root <none> and password file <none>
I20260806 20:25:22.814970 22802 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:22.815060 22802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:22.815349 22802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:22.818321 22802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "72b37f73baf545aface6be4a364a00d2"
format_stamp: "Formatted at 2026-08-06 20:25:22 on dist-test-slave-6k22"
server_key: "34dedaacd6be525c154ecdbd8467ddba"
server_key_iv: "486ef0186b5721fc24b1299eaa43affe"
server_key_version: "encryptionkey@0"
I20260806 20:25:22.819196 22802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "72b37f73baf545aface6be4a364a00d2"
format_stamp: "Formatted at 2026-08-06 20:25:22 on dist-test-slave-6k22"
server_key: "34dedaacd6be525c154ecdbd8467ddba"
server_key_iv: "486ef0186b5721fc24b1299eaa43affe"
server_key_version: "encryptionkey@0"
I20260806 20:25:22.824530 22802 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:25:22.828029 22816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:22.829496 22802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:25:22.829670 22802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "72b37f73baf545aface6be4a364a00d2"
format_stamp: "Formatted at 2026-08-06 20:25:22 on dist-test-slave-6k22"
server_key: "34dedaacd6be525c154ecdbd8467ddba"
server_key_iv: "486ef0186b5721fc24b1299eaa43affe"
server_key_version: "encryptionkey@0"
I20260806 20:25:22.829914 22802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:22.835633 22802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:22.836404 22802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:22.836671 22802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:22.837618 22802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:22.839068 22802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:22.839180 22802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:22.839310 22802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:22.839406 22802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:22.873438 22802 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:34681
I20260806 20:25:22.873518 22930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:34681 every 8 connection(s)
I20260806 20:25:22.874785 22802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:22.876164 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 22802
I20260806 20:25:22.876420 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:22.877229 17847 external_mini_cluster.cc:1428] Setting key 1ef4f086fc9478763f64e797ae4df790
I20260806 20:25:22.881224 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:33685
--builtin_ntp_servers=127.17.109.212:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 20:25:22.894863 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:22.877884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:58483 (local address 127.17.109.254:33685)
0806 20:25:22.891936 (+ 14052us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:22.891945 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:22.891967 (+ 22us) server_negotiation.cc:407] Connection header received
0806 20:25:22.892041 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:22.892048 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:22.892114 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:22.892237 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:22.893908 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:22.893916 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:22.893923 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:22.893948 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:22.894020 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:22.894039 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:22.894182 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:22.894371 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 20:25:22.894448 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13905,"thread_start_us":13817,"threads_started":1}
I20260806 20:25:22.895740 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:22.878520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:33685 (local address 127.17.109.193:58483)
0806 20:25:22.884036 (+ 5516us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:22.884079 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:22.885656 (+ 1577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:22.892584 (+ 6928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:22.892598 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:22.892743 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:22.893531 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:25:22.893533 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:22.893610 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:25:22.893707 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:22.894143 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:22.894150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:22.894156 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 20:25:22.894399 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 20:25:22.894609 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"thread_start_us":373,"threads_started":1}
I20260806 20:25:22.897325 22931 heartbeater.cc:347] Connected to a master server at 127.17.109.254:33685
I20260806 20:25:22.897627 22931 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:22.898236 22931 heartbeater.cc:511] Master 127.17.109.254:33685 requested a full tablet report, sending...
I20260806 20:25:22.900530 22741 ts_manager.cc:194] Registered new tserver with Master: 72b37f73baf545aface6be4a364a00d2 (127.17.109.193:34681)
W20260806 20:25:23.223997 22935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:23.224267 22935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:23.224465 22935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:23.232048 22935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:23.232136 22935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:23.232334 22935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:23.241573 22935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:33685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:23.242832 22935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:23.244731 22935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:23.257700 22940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:23.257860 22941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:23.257879 22943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:23.258996 22935 server_base.cc:1034] running on GCE node
I20260806 20:25:23.259502 22935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:23.260499 22935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:23.261746 22935 hybrid_clock.cc:648] HybridClock initialized: now 1786047923261698 us; error 49 us; skew 500 ppm
I20260806 20:25:23.262049 22935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:23.264529 22935 webserver.cc:460] Webserver started at http://127.17.109.194:42589/ using document root <none> and password file <none>
I20260806 20:25:23.265201 22935 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:23.265331 22935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:23.265664 22935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:23.268630 22935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "7ca304d825924e3dbed47298204f4d1a"
format_stamp: "Formatted at 2026-08-06 20:25:23 on dist-test-slave-6k22"
server_key: "36035ef5292f833c9a2a3500f0dff61f"
server_key_iv: "3a23b58752cf5ee2a808a32b29a521df"
server_key_version: "encryptionkey@0"
I20260806 20:25:23.269553 22935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "7ca304d825924e3dbed47298204f4d1a"
format_stamp: "Formatted at 2026-08-06 20:25:23 on dist-test-slave-6k22"
server_key: "36035ef5292f833c9a2a3500f0dff61f"
server_key_iv: "3a23b58752cf5ee2a808a32b29a521df"
server_key_version: "encryptionkey@0"
I20260806 20:25:23.274955 22935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 20:25:23.278505 22950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:23.279870 22935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 20:25:23.280042 22935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "7ca304d825924e3dbed47298204f4d1a"
format_stamp: "Formatted at 2026-08-06 20:25:23 on dist-test-slave-6k22"
server_key: "36035ef5292f833c9a2a3500f0dff61f"
server_key_iv: "3a23b58752cf5ee2a808a32b29a521df"
server_key_version: "encryptionkey@0"
I20260806 20:25:23.280264 22935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:23.285883 22935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:23.286628 22935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:23.286911 22935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:23.287889 22935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:23.289367 22935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:23.289486 22935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:23.289618 22935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:23.289712 22935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:23.327936 22935 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:45255
I20260806 20:25:23.328025 23071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:45255 every 8 connection(s)
I20260806 20:25:23.329293 22935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:23.330510 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 22935
I20260806 20:25:23.330717 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:23.331313 17847 external_mini_cluster.cc:1428] Setting key 1c2974df0305a916b0001f2adaf5dc35
I20260806 20:25:23.335052 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:33685
--builtin_ntp_servers=127.17.109.212:38947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 20:25:23.339548 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.332271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:60133 (local address 127.17.109.254:33685)
0806 20:25:23.332495 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:23.332502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:23.334360 (+ 1858us) server_negotiation.cc:407] Connection header received
0806 20:25:23.337197 (+ 2837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:23.337201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:23.337277 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:23.337368 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:23.338907 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:23.338910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:23.338914 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:23.338929 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:23.338963 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:23.338975 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:23.339078 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:23.339309 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 20:25:23.339363 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 20:25:23.340957 23074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.332972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:33685 (local address 127.17.109.194:60133)
0806 20:25:23.334193 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:23.334241 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:23.336837 (+ 2596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:23.337663 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:23.337677 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:23.337799 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:23.338551 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:25:23.338557 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:23.338613 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:25:23.338730 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:23.339075 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:23.339088 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:23.339098 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 20:25:23.339390 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 20:25:23.339679 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":373,"threads_started":1}
I20260806 20:25:23.342402 23072 heartbeater.cc:347] Connected to a master server at 127.17.109.254:33685
I20260806 20:25:23.342736 23072 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:23.343333 23072 heartbeater.cc:511] Master 127.17.109.254:33685 requested a full tablet report, sending...
I20260806 20:25:23.345080 22741 ts_manager.cc:194] Registered new tserver with Master: 7ca304d825924e3dbed47298204f4d1a (127.17.109.194:45255)
W20260806 20:25:23.668047 23075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:23.668347 23075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:23.668438 23075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:23.675776 23075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:23.675889 23075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:23.676007 23075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:23.685436 23075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:33685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:23.686731 23075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:23.688503 23075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:23.701808 23080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:23.701835 23081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:23.701956 23084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:23.702493 23075 server_base.cc:1034] running on GCE node
I20260806 20:25:23.703161 23075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:23.704264 23075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:23.705507 23075 hybrid_clock.cc:648] HybridClock initialized: now 1786047923705468 us; error 60 us; skew 500 ppm
I20260806 20:25:23.705852 23075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:23.708274 23075 webserver.cc:460] Webserver started at http://127.17.109.195:37383/ using document root <none> and password file <none>
I20260806 20:25:23.708969 23075 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:23.709105 23075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:23.709448 23075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:23.712467 23075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "2113544ea31344cc9cb2446df2b18f14"
format_stamp: "Formatted at 2026-08-06 20:25:23 on dist-test-slave-6k22"
server_key: "b5be785bd72efda088806f48509b5059"
server_key_iv: "e9d1fde04a1ee5ee14cc6088aa2fdd20"
server_key_version: "encryptionkey@0"
I20260806 20:25:23.713459 23075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "2113544ea31344cc9cb2446df2b18f14"
format_stamp: "Formatted at 2026-08-06 20:25:23 on dist-test-slave-6k22"
server_key: "b5be785bd72efda088806f48509b5059"
server_key_iv: "e9d1fde04a1ee5ee14cc6088aa2fdd20"
server_key_version: "encryptionkey@0"
I20260806 20:25:23.719084 23075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 20:25:23.722784 23090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:23.724400 23075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:23.724591 23075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "2113544ea31344cc9cb2446df2b18f14"
format_stamp: "Formatted at 2026-08-06 20:25:23 on dist-test-slave-6k22"
server_key: "b5be785bd72efda088806f48509b5059"
server_key_iv: "e9d1fde04a1ee5ee14cc6088aa2fdd20"
server_key_version: "encryptionkey@0"
I20260806 20:25:23.724830 23075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:23.730849 23075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:23.731734 23075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:23.732005 23075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:23.733004 23075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:23.734635 23075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:23.734754 23075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:23.734896 23075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:23.734985 23075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:23.771097 23075 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:32779
I20260806 20:25:23.771119 23211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:32779 every 8 connection(s)
I20260806 20:25:23.772635 23075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:23.780972 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.775381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:50903 (local address 127.17.109.254:33685)
0806 20:25:23.775646 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:23.775653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:23.777139 (+ 1486us) server_negotiation.cc:407] Connection header received
0806 20:25:23.778829 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:23.778833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:23.778886 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:23.778983 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:23.780351 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:23.780355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:23.780358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:23.780374 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:23.780405 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:23.780417 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:23.780520 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:23.780680 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 20:25:23.780752 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 20:25:23.781523 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 23075
I20260806 20:25:23.781710 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:23.782004 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.776030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:33685 (local address 127.17.109.195:50903)
0806 20:25:23.776982 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:23.777019 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:23.778556 (+ 1537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:23.779204 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:23.779212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:23.779307 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:23.780066 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:25:23.780069 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:23.780106 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:25:23.780196 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:23.780506 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:23.780513 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:23.780518 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 20:25:23.780727 (+ 209us) client_negotiation.cc:240] Negotiation successful
0806 20:25:23.780914 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":287,"threads_started":1}
I20260806 20:25:23.782182 17847 external_mini_cluster.cc:1428] Setting key 9f945271fd04d78aa2aa45627ab17a73
I20260806 20:25:23.783577 23212 heartbeater.cc:347] Connected to a master server at 127.17.109.254:33685
I20260806 20:25:23.783946 23212 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:23.784721 23212 heartbeater.cc:511] Master 127.17.109.254:33685 requested a full tablet report, sending...
I20260806 20:25:23.786080 22741 ts_manager.cc:194] Registered new tserver with Master: 2113544ea31344cc9cb2446df2b18f14 (127.17.109.195:32779)
I20260806 20:25:23.797385 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:25:23.804540 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.801970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44058 (local address 127.17.109.254:33685)
0806 20:25:23.802131 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:23.802136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:23.802664 (+ 528us) server_negotiation.cc:407] Connection header received
0806 20:25:23.802827 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:23.802831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:23.802909 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:23.802993 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:23.803938 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:23.803942 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:23.803945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:23.803961 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:23.803993 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:23.804004 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:23.804094 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:23.804265 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 20:25:23.804328 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 20:25:23.808703 22741 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:25:23.815821 22741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:23.844153 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.829334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54330 (local address 127.17.109.195:32779)
0806 20:25:23.839573 (+ 10239us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:23.839582 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:23.840154 (+ 572us) server_negotiation.cc:407] Connection header received
0806 20:25:23.840420 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:23.840431 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:23.840644 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:23.840750 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:23.843074 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:23.843084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:23.843100 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:23.843121 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:23.843187 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:23.843208 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:23.843391 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:23.843592 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 20:25:23.843726 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10005,"thread_start_us":9832,"threads_started":1}
I20260806 20:25:23.844550 23225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.828495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45255 (local address 127.0.0.1:59992)
0806 20:25:23.839045 (+ 10550us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:23.839083 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:23.840058 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:23.842073 (+ 2015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:23.842080 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:23.842112 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:23.843213 (+ 1101us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:25:23.843216 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:23.843228 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:25:23.843290 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:23.843857 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:23.843862 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:23.843866 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 20:25:23.844004 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 20:25:23.844101 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10301,"thread_start_us":11230,"threads_started":1}
I20260806 20:25:23.845163 23224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.828717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:32779 (local address 127.0.0.1:54330)
0806 20:25:23.840035 (+ 11318us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:23.840054 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:23.840246 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:23.841807 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:23.841814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:23.841917 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:23.842776 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:25:23.842781 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:23.842828 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:25:23.842900 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:23.843354 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:23.843368 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:23.843379 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 20:25:23.843996 (+ 617us) client_negotiation.cc:240] Negotiation successful
0806 20:25:23.844128 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11167,"mutex_wait_us":36,"thread_start_us":9932,"threads_started":1}
I20260806 20:25:23.845469 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.828828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59992 (local address 127.17.109.194:45255)
0806 20:25:23.830045 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:23.830057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:23.839223 (+ 9166us) server_negotiation.cc:407] Connection header received
0806 20:25:23.840291 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:23.840304 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:23.840604 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:23.840750 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:23.843498 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:23.843506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:23.843521 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:23.843578 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:23.843646 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:23.843671 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:23.843800 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:23.845011 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0806 20:25:23.845111 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":903,"threads_started":1}
I20260806 20:25:23.846621 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.832619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58594)
0806 20:25:23.842948 (+ 10329us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:23.842966 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:23.843063 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:23.844742 (+ 1679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:23.844747 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:23.844769 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:23.845535 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:25:23.845541 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:23.845559 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:25:23.845601 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:23.845989 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:23.845994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:23.845998 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 20:25:23.846121 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 20:25:23.846171 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10190,"spinlock_wait_cycles":2304,"thread_start_us":10164,"threads_started":1}
I20260806 20:25:23.848869 23002 tablet_service.cc:1467] Processing CreateTablet for tablet bb2e4b07e32a40ca9f5500829f522132 (DEFAULT_TABLE table=test-table [id=006319859ff64934bb40c6daf033aa0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:23.848861 23142 tablet_service.cc:1467] Processing CreateTablet for tablet bb2e4b07e32a40ca9f5500829f522132 (DEFAULT_TABLE table=test-table [id=006319859ff64934bb40c6daf033aa0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:23.849777 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:23.833254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.17.109.193:34681)
0806 20:25:23.843329 (+ 10075us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:23.843340 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:23.843385 (+ 45us) server_negotiation.cc:407] Connection header received
0806 20:25:23.843458 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:23.843463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:23.843727 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:23.843873 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:23.845766 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:23.845774 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:23.845786 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:23.845807 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:23.845866 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:23.845885 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:23.846026 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:23.846106 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 20:25:23.846200 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9873,"thread_start_us":9320,"threads_started":1}
I20260806 20:25:23.850178 23002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb2e4b07e32a40ca9f5500829f522132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:23.850170 23142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb2e4b07e32a40ca9f5500829f522132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:23.853907 22864 tablet_service.cc:1467] Processing CreateTablet for tablet bb2e4b07e32a40ca9f5500829f522132 (DEFAULT_TABLE table=test-table [id=006319859ff64934bb40c6daf033aa0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:23.855290 22864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb2e4b07e32a40ca9f5500829f522132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:23.861452 23230 tablet_bootstrap.cc:492] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Bootstrap starting.
I20260806 20:25:23.864960 23231 tablet_bootstrap.cc:492] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Bootstrap starting.
I20260806 20:25:23.865686 23230 tablet_bootstrap.cc:654] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:23.867077 23230 log.cc:824] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:23.868573 23231 tablet_bootstrap.cc:654] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:23.869357 23233 tablet_bootstrap.cc:492] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Bootstrap starting.
I20260806 20:25:23.869796 23231 log.cc:824] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:23.870801 23230 tablet_bootstrap.cc:492] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: No bootstrap required, opened a new log
I20260806 20:25:23.871074 23230 ts_tablet_manager.cc:1397] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260806 20:25:23.873061 23231 tablet_bootstrap.cc:492] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: No bootstrap required, opened a new log
I20260806 20:25:23.873262 23233 tablet_bootstrap.cc:654] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:23.873350 23231 ts_tablet_manager.cc:1397] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260806 20:25:23.874624 23233 log.cc:824] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:23.877825 23233 tablet_bootstrap.cc:492] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: No bootstrap required, opened a new log
I20260806 20:25:23.878129 23233 ts_tablet_manager.cc:1397] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 20:25:23.884491 23231 raft_consensus.cc:348] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:23.885037 23231 raft_consensus.cc:374] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:23.885154 23231 raft_consensus.cc:729] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ca304d825924e3dbed47298204f4d1a, State: Initialized, Role: FOLLOWER
I20260806 20:25:23.885030 23230 raft_consensus.cc:348] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:23.885516 23230 raft_consensus.cc:374] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:23.885644 23230 raft_consensus.cc:729] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2113544ea31344cc9cb2446df2b18f14, State: Initialized, Role: FOLLOWER
I20260806 20:25:23.886003 23231 consensus_queue.cc:260] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:23.886361 23230 consensus_queue.cc:260] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:23.887815 23072 heartbeater.cc:503] Master 127.17.109.254:33685 was elected leader, sending a full tablet report...
I20260806 20:25:23.888813 23231 ts_tablet_manager.cc:1428] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260806 20:25:23.889954 23212 heartbeater.cc:503] Master 127.17.109.254:33685 was elected leader, sending a full tablet report...
I20260806 20:25:23.890748 23230 ts_tablet_manager.cc:1428] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Time spent starting tablet: real 0.020s user 0.009s sys 0.007s
I20260806 20:25:23.891631 23233 raft_consensus.cc:348] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:23.892211 23233 raft_consensus.cc:374] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:23.892335 23233 raft_consensus.cc:729] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b37f73baf545aface6be4a364a00d2, State: Initialized, Role: FOLLOWER
I20260806 20:25:23.893081 23233 consensus_queue.cc:260] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:23.894372 22931 heartbeater.cc:503] Master 127.17.109.254:33685 was elected leader, sending a full tablet report...
I20260806 20:25:23.895045 23233 ts_tablet_manager.cc:1428] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Time spent starting tablet: real 0.017s user 0.013s sys 0.000s
W20260806 20:25:24.024475 23213 tablet.cc:2369] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:25:24.049176 23238 raft_consensus.cc:482] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:24.049535 23238 raft_consensus.cc:504] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:24.051468 23238 leader_election.cc:288] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ca304d825924e3dbed47298204f4d1a (127.17.109.194:45255), 72b37f73baf545aface6be4a364a00d2 (127.17.109.193:34681)
I20260806 20:25:24.054625 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.051703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45255 (local address 127.17.109.195:33045)
0806 20:25:24.051924 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.051954 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:24.052072 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:24.052620 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:24.052628 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:24.052651 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:24.053796 (+ 1145us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:25:24.053799 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:24.053814 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:25:24.053855 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:24.054187 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:24.054196 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:24.054205 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 20:25:24.054323 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 20:25:24.054369 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 20:25:24.054728 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.051893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:33045 (local address 127.17.109.194:45255)
0806 20:25:24.052119 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:24.052127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:24.052153 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:25:24.052229 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:24.052235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:24.052303 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:24.052420 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:24.054007 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:24.054012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:24.054015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:24.054031 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:24.054066 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:24.054079 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:24.054185 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:24.054337 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 20:25:24.054402 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 20:25:24.055034 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.052300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:34481 (local address 127.17.109.193:34681)
0806 20:25:24.052502 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:24.052508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:24.052972 (+ 464us) server_negotiation.cc:407] Connection header received
0806 20:25:24.053110 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:24.053114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:24.053166 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:24.053304 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:24.054351 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:24.054356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:24.054361 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:24.054380 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:24.054415 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:24.054430 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:24.054550 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:24.054692 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 20:25:24.054761 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 20:25:24.055289 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.052041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.17.109.195:34481)
0806 20:25:24.052849 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.052864 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:24.052991 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:24.053351 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:24.053357 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:24.053385 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:24.054082 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:25:24.054084 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:24.054096 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:25:24.054157 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:24.054543 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:24.054548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:24.054553 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 20:25:24.054701 (+ 148us) client_negotiation.cc:240] Negotiation successful
0806 20:25:24.054762 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":448,"threads_started":1}
I20260806 20:25:24.055989 23022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb2e4b07e32a40ca9f5500829f522132" candidate_uuid: "2113544ea31344cc9cb2446df2b18f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca304d825924e3dbed47298204f4d1a" is_pre_election: true
I20260806 20:25:24.056478 22885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb2e4b07e32a40ca9f5500829f522132" candidate_uuid: "2113544ea31344cc9cb2446df2b18f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72b37f73baf545aface6be4a364a00d2" is_pre_election: true
I20260806 20:25:24.056670 23022 raft_consensus.cc:2454] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2113544ea31344cc9cb2446df2b18f14 in term 0.
I20260806 20:25:24.057113 22885 raft_consensus.cc:2454] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2113544ea31344cc9cb2446df2b18f14 in term 0.
I20260806 20:25:24.057454 23093 leader_election.cc:302] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2113544ea31344cc9cb2446df2b18f14, 7ca304d825924e3dbed47298204f4d1a; no voters:
I20260806 20:25:24.057933 23238 raft_consensus.cc:2781] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:24.058063 23238 raft_consensus.cc:482] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:24.058225 23238 raft_consensus.cc:3037] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:24.060592 23238 raft_consensus.cc:504] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:24.061228 23238 leader_election.cc:288] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [CANDIDATE]: Term 1 election: Requested vote from peers 7ca304d825924e3dbed47298204f4d1a (127.17.109.194:45255), 72b37f73baf545aface6be4a364a00d2 (127.17.109.193:34681)
I20260806 20:25:24.061745 22885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb2e4b07e32a40ca9f5500829f522132" candidate_uuid: "2113544ea31344cc9cb2446df2b18f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72b37f73baf545aface6be4a364a00d2"
I20260806 20:25:24.061775 23022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb2e4b07e32a40ca9f5500829f522132" candidate_uuid: "2113544ea31344cc9cb2446df2b18f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ca304d825924e3dbed47298204f4d1a"
I20260806 20:25:24.061947 22885 raft_consensus.cc:3037] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:24.062044 23022 raft_consensus.cc:3037] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:24.064364 22885 raft_consensus.cc:2454] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2113544ea31344cc9cb2446df2b18f14 in term 1.
I20260806 20:25:24.064436 23022 raft_consensus.cc:2454] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2113544ea31344cc9cb2446df2b18f14 in term 1.
I20260806 20:25:24.064885 23092 leader_election.cc:302] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2113544ea31344cc9cb2446df2b18f14, 72b37f73baf545aface6be4a364a00d2; no voters:
I20260806 20:25:24.065193 23238 raft_consensus.cc:2781] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:24.065706 23238 raft_consensus.cc:686] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 1 LEADER]: Becoming Leader. State: Replica: 2113544ea31344cc9cb2446df2b18f14, State: Running, Role: LEADER
I20260806 20:25:24.066195 23238 consensus_queue.cc:237] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } }
I20260806 20:25:24.074337 22741 catalog_manager.cc:5495] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2113544ea31344cc9cb2446df2b18f14 (127.17.109.195). New cstate: current_term: 1 leader_uuid: "2113544ea31344cc9cb2446df2b18f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2113544ea31344cc9cb2446df2b18f14" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 32779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 } health_report { overall_health: UNKNOWN } } }
W20260806 20:25:24.081460 23073 tablet.cc:2369] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:25:24.127321 22932 tablet.cc:2369] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:25:24.138922 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.136190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.17.109.195:32779)
0806 20:25:24.136403 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:24.136408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:24.136518 (+ 110us) server_negotiation.cc:407] Connection header received
0806 20:25:24.136761 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:24.136766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:24.136859 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:24.136949 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:24.138105 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:24.138109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:24.138112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:24.138130 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:24.138164 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:24.138176 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:24.138299 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:24.138568 (+ 269us) server_negotiation.cc:299] Negotiation successful
0806 20:25:24.138643 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:25:24.151737 22885 raft_consensus.cc:1261] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 1 FOLLOWER]: Refusing update from remote peer 2113544ea31344cc9cb2446df2b18f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:24.151750 23022 raft_consensus.cc:1261] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 1 FOLLOWER]: Refusing update from remote peer 2113544ea31344cc9cb2446df2b18f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:24.152876 23241 consensus_queue.cc:1035] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72b37f73baf545aface6be4a364a00d2" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:24.153390 23238 consensus_queue.cc:1035] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ca304d825924e3dbed47298204f4d1a" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:24.175352 23246 mvcc.cc:205] Tried to move back new op lower bound from 7315652297311772672 to 7315652296979222528. Current Snapshot: MvccSnapshot[applied={T|T < 7315652297311772672}]
I20260806 20:25:24.184448 22741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:24.184821 22741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:24.189136 22741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2c669d142c92407aad194851743c5172: Sending DeleteTablet for 3 replicas of tablet bb2e4b07e32a40ca9f5500829f522132
I20260806 20:25:24.190363 23002 tablet_service.cc:1514] Processing DeleteTablet for tablet bb2e4b07e32a40ca9f5500829f522132 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:24 UTC) from {username='slave'} at 127.0.0.1:59992
I20260806 20:25:24.190862 23142 tablet_service.cc:1514] Processing DeleteTablet for tablet bb2e4b07e32a40ca9f5500829f522132 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:24 UTC) from {username='slave'} at 127.0.0.1:54330
I20260806 20:25:24.191370 23253 tablet_replica.cc:330] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: stopping tablet replica
I20260806 20:25:24.191783 23254 tablet_replica.cc:330] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: stopping tablet replica
I20260806 20:25:24.191895 23253 raft_consensus.cc:2229] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:24.192051 22864 tablet_service.cc:1514] Processing DeleteTablet for tablet bb2e4b07e32a40ca9f5500829f522132 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:24 UTC) from {username='slave'} at 127.0.0.1:58594
I20260806 20:25:24.192175 23254 raft_consensus.cc:2229] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:24.192451 23253 raft_consensus.cc:2258] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:24.192917 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 22802
I20260806 20:25:24.192878 23254 raft_consensus.cc:2258] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:24.195140 23253 ts_tablet_manager.cc:1905] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:24.195626 23254 ts_tablet_manager.cc:1905] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:24.203702 23253 ts_tablet_manager.cc:1918] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:24.203886 23253 log.cc:1196] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/bb2e4b07e32a40ca9f5500829f522132
I20260806 20:25:24.204545 23253 ts_tablet_manager.cc:1939] T bb2e4b07e32a40ca9f5500829f522132 P 7ca304d825924e3dbed47298204f4d1a: Deleting consensus metadata
I20260806 20:25:24.205893 23255 tablet_replica.cc:330] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: stopping tablet replica
I20260806 20:25:24.206255 23255 raft_consensus.cc:2229] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:24.206724 23255 raft_consensus.cc:2258] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:24.206960 22726 catalog_manager.cc:4841] TS 7ca304d825924e3dbed47298204f4d1a (127.17.109.194:45255): tablet bb2e4b07e32a40ca9f5500829f522132 (table test-table [id=006319859ff64934bb40c6daf033aa0a]) successfully deleted
I20260806 20:25:24.208150 23254 ts_tablet_manager.cc:1918] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:24.208343 23254 log.cc:1196] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/bb2e4b07e32a40ca9f5500829f522132
I20260806 20:25:24.208912 23254 ts_tablet_manager.cc:1939] T bb2e4b07e32a40ca9f5500829f522132 P 2113544ea31344cc9cb2446df2b18f14: Deleting consensus metadata
I20260806 20:25:24.210565 22727 catalog_manager.cc:4841] TS 2113544ea31344cc9cb2446df2b18f14 (127.17.109.195:32779): tablet bb2e4b07e32a40ca9f5500829f522132 (table test-table [id=006319859ff64934bb40c6daf033aa0a]) successfully deleted
I20260806 20:25:24.211018 23255 ts_tablet_manager.cc:1905] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:24.217377 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.194762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58598 (local address 127.17.109.193:34681)
0806 20:25:24.194942 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:24.194948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:24.208972 (+ 14024us) server_negotiation.cc:407] Connection header received
0806 20:25:24.209536 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:24.209542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:24.209632 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:24.209724 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:24.216398 (+ 6674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:24.216404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:24.216413 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:24.216437 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:24.216492 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:24.216511 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:24.216671 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:24.216916 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 20:25:24.216991 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:25:24.218417 23255 ts_tablet_manager.cc:1918] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:24.218576 22926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:24.218602 23255 log.cc:1196] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/bb2e4b07e32a40ca9f5500829f522132
I20260806 20:25:24.376359 23255 ts_tablet_manager.cc:1939] T bb2e4b07e32a40ca9f5500829f522132 P 72b37f73baf545aface6be4a364a00d2: Deleting consensus metadata
I20260806 20:25:24.376521 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 22802
I20260806 20:25:24.396688 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 22935
W20260806 20:25:24.396605 22725 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.109.193:34681 (error 108)
I20260806 20:25:24.399216 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.398263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58610)
0806 20:25:24.398791 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.398998 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":274,"threads_started":1}
W20260806 20:25:24.399435 22725 catalog_manager.cc:4584] TS 72b37f73baf545aface6be4a364a00d2 (127.17.109.193:34681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb2e4b07e32a40ca9f5500829f522132: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
I20260806 20:25:24.401021 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.398017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60006 (local address 127.17.109.194:45255)
0806 20:25:24.398265 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:24.398273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:24.398298 (+ 25us) server_negotiation.cc:407] Connection header received
0806 20:25:24.398393 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:24.398401 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:24.398525 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:24.398660 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:24.400263 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:24.400270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:24.400277 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:24.400302 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:24.400350 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:24.400371 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:24.400527 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:24.400585 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 20:25:24.400657 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 20:25:24.401799 23067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:24.453581 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.452904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58616)
0806 20:25:24.453213 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.453352 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260806 20:25:24.454983 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.454534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58620)
0806 20:25:24.454714 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.454811 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 20:25:24.494499 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 22935
I20260806 20:25:24.515499 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.515047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58626)
0806 20:25:24.515256 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.515355 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 20:25:24.516677 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.516267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58640)
0806 20:25:24.516436 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.516540 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 20:25:24.524431 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 23075
I20260806 20:25:24.527938 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.525289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54342 (local address 127.17.109.195:32779)
0806 20:25:24.525529 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:24.525536 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:24.525558 (+ 22us) server_negotiation.cc:407] Connection header received
0806 20:25:24.525713 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:24.525718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:24.525775 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:24.525916 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:24.527018 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:24.527026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:24.527036 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:24.527060 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:24.527098 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:24.527111 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:24.527272 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:24.527445 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 20:25:24.527516 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 20:25:24.528656 23207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:24.597644 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.597105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58644)
0806 20:25:24.597395 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.597495 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260806 20:25:24.598860 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.598417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58658)
0806 20:25:24.598612 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.598705 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260806 20:25:24.650194 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 23075
I20260806 20:25:24.671993 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 22709
I20260806 20:25:24.673441 22772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:24.782025 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 22709
I20260806 20:25:24.782186 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:24.780829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34681 (local address 127.0.0.1:58666)
0806 20:25:24.781802 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:24.781951 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
2026-08-06T20:25:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3041 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 20:25:24.829434 17847 mini_oidc.cc:152] Starting JWKS server
I20260806 20:25:24.834519 17847 webserver.cc:460] Webserver started at https://127.0.0.1:35439/ using document root <none> and password file <none>
I20260806 20:25:24.834944 17847 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 20:25:24.835824 17847 webserver.cc:460] Webserver started at http://127.0.0.1:41683/ using document root <none> and password file <none>
2026-08-06T20:25:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:24Z Disabled control of system clock
I20260806 20:25:24.871516 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:42041
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:42041
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35439/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/testchainca.cert with env {}
W20260806 20:25:25.200519 23268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:25.200917 23268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:25.201090 23268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:25.209259 23268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:25.209431 23268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:25.209535 23268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:25.209661 23268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:25.209748 23268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 20:25:25.209874 23268 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 20:25:25.209962 23268 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35439/jwks.json
I20260806 20:25:25.221568 23268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:42041
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:42041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35439/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:25.223078 23268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:25.224589 23268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:25.238268 23273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:25.238204 23274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:25.238214 23276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:25.239215 23268 server_base.cc:1034] running on GCE node
I20260806 20:25:25.240196 23268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:25.241706 23268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:25.242967 23268 hybrid_clock.cc:648] HybridClock initialized: now 1786047925242921 us; error 40 us; skew 500 ppm
I20260806 20:25:25.243355 23268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:25.246022 23268 webserver.cc:460] Webserver started at http://127.17.109.254:33071/ using document root <none> and password file <none>
I20260806 20:25:25.246785 23268 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:25.246932 23268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:25.247262 23268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:25.250576 23268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "df3e519753ec43a5b62574ff4203f81f"
format_stamp: "Formatted at 2026-08-06 20:25:25 on dist-test-slave-6k22"
server_key: "4aff72cf11874aab13daf5bbab7e57a3"
server_key_iv: "93ef65339e300b2dbd81f849bedc0aa8"
server_key_version: "encryptionkey@0"
I20260806 20:25:25.251629 23268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "df3e519753ec43a5b62574ff4203f81f"
format_stamp: "Formatted at 2026-08-06 20:25:25 on dist-test-slave-6k22"
server_key: "4aff72cf11874aab13daf5bbab7e57a3"
server_key_iv: "93ef65339e300b2dbd81f849bedc0aa8"
server_key_version: "encryptionkey@0"
I20260806 20:25:25.257164 23268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 20:25:25.261257 23283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:25.262670 23268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:25:25.262915 23268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "df3e519753ec43a5b62574ff4203f81f"
format_stamp: "Formatted at 2026-08-06 20:25:25 on dist-test-slave-6k22"
server_key: "4aff72cf11874aab13daf5bbab7e57a3"
server_key_iv: "93ef65339e300b2dbd81f849bedc0aa8"
server_key_version: "encryptionkey@0"
I20260806 20:25:25.263144 23268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:25.289043 23268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:25.289951 23268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:25.290302 23268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:25.310787 23268 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:42041
I20260806 20:25:25.310829 23336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:42041 every 8 connection(s)
I20260806 20:25:25.312332 23268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:25.312953 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 23268
I20260806 20:25:25.313210 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:25.314078 17847 external_mini_cluster.cc:1428] Setting key 60d558e53bad608139f0df9181547d89
I20260806 20:25:25.317276 23337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:25.328377 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:25.316296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38240 (local address 127.17.109.254:42041)
0806 20:25:25.317294 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:25.317305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:25.317347 (+ 42us) server_negotiation.cc:407] Connection header received
0806 20:25:25.319057 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:25.319077 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:25.319534 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:25.320015 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:25.320604 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:25.321942 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:25.323454 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:25.324197 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:25.324436 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:25.325769 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:25.325777 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:25.325791 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:25.326515 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:25.326598 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:25.326623 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:25.326767 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:25.327022 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 20:25:25.327343 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":365,"threads_started":1}
I20260806 20:25:25.329025 23337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f: Bootstrap starting.
I20260806 20:25:25.332284 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:25:25.332615 23337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:25.333634 23337 log.cc:824] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:25.336742 23337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f: No bootstrap required, opened a new log
I20260806 20:25:25.344098 23337 raft_consensus.cc:348] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3e519753ec43a5b62574ff4203f81f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42041 } }
I20260806 20:25:25.344460 23337 raft_consensus.cc:374] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:25.344551 23337 raft_consensus.cc:729] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df3e519753ec43a5b62574ff4203f81f, State: Initialized, Role: FOLLOWER
I20260806 20:25:25.345201 23337 consensus_queue.cc:260] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3e519753ec43a5b62574ff4203f81f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42041 } }
I20260806 20:25:25.345467 23337 raft_consensus.cc:388] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:25.345618 23337 raft_consensus.cc:482] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:25.345782 23337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:25.348016 23337 raft_consensus.cc:504] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3e519753ec43a5b62574ff4203f81f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42041 } }
I20260806 20:25:25.348541 23337 leader_election.cc:302] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df3e519753ec43a5b62574ff4203f81f; no voters:
I20260806 20:25:25.349115 23337 leader_election.cc:288] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:25.349308 23342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:25.350113 23342 raft_consensus.cc:686] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [term 1 LEADER]: Becoming Leader. State: Replica: df3e519753ec43a5b62574ff4203f81f, State: Running, Role: LEADER
I20260806 20:25:25.350656 23342 consensus_queue.cc:237] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3e519753ec43a5b62574ff4203f81f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42041 } }
I20260806 20:25:25.351305 23337 sys_catalog.cc:564] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:25.353611 23344 sys_catalog.cc:455] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [sys.catalog]: SysCatalogTable state changed. Reason: New leader df3e519753ec43a5b62574ff4203f81f. Latest consensus state: current_term: 1 leader_uuid: "df3e519753ec43a5b62574ff4203f81f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3e519753ec43a5b62574ff4203f81f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42041 } } }
I20260806 20:25:25.354045 23344 sys_catalog.cc:458] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:25.354935 23343 sys_catalog.cc:455] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df3e519753ec43a5b62574ff4203f81f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3e519753ec43a5b62574ff4203f81f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42041 } } }
I20260806 20:25:25.355450 23343 sys_catalog.cc:458] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:25.360441 23348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:25.366430 23348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:25.375928 23348 catalog_manager.cc:1347] Generated new cluster ID: f34bc25011554514b9a3799b797b1889
I20260806 20:25:25.376070 23348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:25.418007 23348 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:25.420312 23348 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:25.443171 23348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df3e519753ec43a5b62574ff4203f81f: Generated new TSK 0
I20260806 20:25:25.444834 23348 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 20:25:25.499475 23368 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 20:25:25.500202 23368 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 20:25:25.503592 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:25.470466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38248 (local address 127.17.109.254:42041)
0806 20:25:25.470867 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:25.470880 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:25.471890 (+ 1010us) server_negotiation.cc:407] Connection header received
0806 20:25:25.472527 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:25.472532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:25.472670 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:25.472951 (+ 281us) server_negotiation.cc:226] Negotiated authn=JWT
0806 20:25:25.473882 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:25.475015 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:25.477513 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:25.478297 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:25.478551 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:25.479475 (+ 924us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 20:25:25.502085 (+ 22610us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35439/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 20:25:25.502837 (+ 752us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38248: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35439/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":107}
W20260806 20:25:25.504251 23340 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38248: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35439/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 20:25:25.506806 23367 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:25.470298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:42041 (local address 127.0.0.1:38248)
0806 20:25:25.471720 (+ 1422us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:25.471758 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:25.472070 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:25.472999 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:25.473006 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:25.473039 (+ 33us) client_negotiation.cc:189] Negotiated authn=JWT
0806 20:25:25.473614 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:25.473631 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:25.475262 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:25.475272 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:25.477258 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:25.477280 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:25.478648 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:25.478659 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:25.478891 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:25.478985 (+ 94us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 20:25:25.502632 (+ 23647us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35439/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 20:25:25.503328 (+ 696us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.109.254:42041: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35439/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":1080,"thread_start_us":881,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.109.254:42041: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35439/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 20:25:25.511282 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 23268
I20260806 20:25:25.513437 23332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:25.650069 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 23268
2026-08-06T20:25:25Z chronyd exiting
I20260806 20:25:25.717913 17847 test_util.cc:175] -----------------------------------------------
I20260806 20:25:25.718150 17847 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786047878953513-17847-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (893 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 20:25:25.724897 17847 mini_oidc.cc:152] Starting JWKS server
I20260806 20:25:25.728211 17847 webserver.cc:460] Webserver started at https://127.0.0.1:40381/ using document root <none> and password file <none>
I20260806 20:25:25.728674 17847 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 20:25:25.729771 17847 webserver.cc:460] Webserver started at http://127.0.0.1:38227/ using document root <none> and password file <none>
2026-08-06T20:25:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:25Z Disabled control of system clock
I20260806 20:25:25.794924 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38199
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:37163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:38199
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40381/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/testchainca.cert with env {}
W20260806 20:25:26.377398 23375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:26.377995 23375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:26.378275 23375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:26.391379 23375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:26.391676 23375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:26.391793 23375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:26.391994 23375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:26.392115 23375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 20:25:26.392299 23375 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 20:25:26.392437 23375 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40381/jwks.json
I20260806 20:25:26.411309 23375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:38199
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40381/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:26.413869 23375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:26.416293 23375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:26.437052 23382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:26.437063 23385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:26.437175 23383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:26.443212 23375 server_base.cc:1034] running on GCE node
I20260806 20:25:26.444824 23375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:26.447414 23375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:26.448832 23375 hybrid_clock.cc:648] HybridClock initialized: now 1786047926448738 us; error 69 us; skew 500 ppm
I20260806 20:25:26.449702 23375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:26.454138 23375 webserver.cc:460] Webserver started at http://127.17.109.254:46697/ using document root <none> and password file <none>
I20260806 20:25:26.455626 23375 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:26.455899 23375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:26.456547 23375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:26.462183 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "ad17d203b4024259bc8284d49918ef68"
format_stamp: "Formatted at 2026-08-06 20:25:26 on dist-test-slave-6k22"
server_key: "35585f029fb3f1ab32b4831645631522"
server_key_iv: "d386943a07424546faf2ece9e162c232"
server_key_version: "encryptionkey@0"
I20260806 20:25:26.463665 23375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "ad17d203b4024259bc8284d49918ef68"
format_stamp: "Formatted at 2026-08-06 20:25:26 on dist-test-slave-6k22"
server_key: "35585f029fb3f1ab32b4831645631522"
server_key_iv: "d386943a07424546faf2ece9e162c232"
server_key_version: "encryptionkey@0"
I20260806 20:25:26.473631 23375 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.001s
I20260806 20:25:26.481693 23391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:26.484477 23375 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.001s
I20260806 20:25:26.484859 23375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "ad17d203b4024259bc8284d49918ef68"
format_stamp: "Formatted at 2026-08-06 20:25:26 on dist-test-slave-6k22"
server_key: "35585f029fb3f1ab32b4831645631522"
server_key_iv: "d386943a07424546faf2ece9e162c232"
server_key_version: "encryptionkey@0"
I20260806 20:25:26.485318 23375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:26.524125 23375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:26.525743 23375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:26.526273 23375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:26.558744 23375 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:38199
I20260806 20:25:26.558787 23444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:38199 every 8 connection(s)
I20260806 20:25:26.561144 23375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:26.567092 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 23375
I20260806 20:25:26.567524 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:26.569348 23445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:26.569722 17847 external_mini_cluster.cc:1428] Setting key 1f727528b599db81189ea93c6f493f08
I20260806 20:25:26.594067 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:26.573173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60888 (local address 127.17.109.254:38199)
0806 20:25:26.574978 (+ 1805us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:26.575006 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:26.575071 (+ 65us) server_negotiation.cc:407] Connection header received
0806 20:25:26.578219 (+ 3148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:26.578262 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:26.578980 (+ 718us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:26.579397 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:26.580732 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:26.582721 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:26.586026 (+ 3305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:26.587326 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:26.587549 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:26.589915 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:26.589925 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:26.589951 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:26.590928 (+ 977us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:26.591062 (+ 134us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:26.591103 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:26.591269 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:26.591709 (+ 440us) server_negotiation.cc:299] Negotiation successful
0806 20:25:26.592272 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":642,"threads_started":1}
I20260806 20:25:26.595386 23445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68: Bootstrap starting.
I20260806 20:25:26.601110 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:25:26.603706 23445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:26.606290 23445 log.cc:824] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:26.611981 23445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68: No bootstrap required, opened a new log
I20260806 20:25:26.625870 23445 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad17d203b4024259bc8284d49918ef68" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38199 } }
I20260806 20:25:26.626560 23445 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:26.626752 23445 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad17d203b4024259bc8284d49918ef68, State: Initialized, Role: FOLLOWER
I20260806 20:25:26.627863 23445 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad17d203b4024259bc8284d49918ef68" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38199 } }
I20260806 20:25:26.628202 23445 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:26.628477 23445 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:26.628798 23445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:26.632920 23445 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad17d203b4024259bc8284d49918ef68" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38199 } }
I20260806 20:25:26.633721 23445 leader_election.cc:302] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad17d203b4024259bc8284d49918ef68; no voters:
I20260806 20:25:26.634523 23445 leader_election.cc:288] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:26.634893 23450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:26.637521 23450 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [term 1 LEADER]: Becoming Leader. State: Replica: ad17d203b4024259bc8284d49918ef68, State: Running, Role: LEADER
I20260806 20:25:26.638501 23445 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:26.638453 23450 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad17d203b4024259bc8284d49918ef68" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38199 } }
I20260806 20:25:26.644703 23452 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad17d203b4024259bc8284d49918ef68. Latest consensus state: current_term: 1 leader_uuid: "ad17d203b4024259bc8284d49918ef68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad17d203b4024259bc8284d49918ef68" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38199 } } }
I20260806 20:25:26.645493 23452 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:26.646637 23451 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad17d203b4024259bc8284d49918ef68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad17d203b4024259bc8284d49918ef68" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38199 } } }
I20260806 20:25:26.647313 23451 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:26.649974 23458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:26.660426 23458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:26.676123 23458 catalog_manager.cc:1347] Generated new cluster ID: 4fbc26b9c14d4c06a5fc25a0b5cfea87
I20260806 20:25:26.676376 23458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:26.703616 23458 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:26.705677 23458 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:26.719467 23458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad17d203b4024259bc8284d49918ef68: Generated new TSK 0
I20260806 20:25:26.720875 23458 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 20:25:26.774156 23475 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 20:25:26.774381 23475 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260806 20:25:26.776882 23474 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:26.745514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38199 (local address 127.0.0.1:60890)
0806 20:25:26.746631 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:26.746649 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:26.746836 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:26.747849 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:26.747853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:26.747864 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0806 20:25:26.748287 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:26.748305 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:26.750668 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:26.750673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:26.752386 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:26.752407 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:26.754181 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:26.754193 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:26.754412 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:26.754464 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 20:25:26.776093 (+ 21629us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 20:25:26.776412 (+ 319us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.109.254:38199: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":885,"thread_start_us":617,"threads_started":1}
I20260806 20:25:26.777197 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:26.745801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60890 (local address 127.17.109.254:38199)
0806 20:25:26.746103 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:26.746113 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:26.746763 (+ 650us) server_negotiation.cc:407] Connection header received
0806 20:25:26.747123 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:26.747140 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:26.747361 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:26.747833 (+ 472us) server_negotiation.cc:226] Negotiated authn=JWT
0806 20:25:26.748617 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:26.750479 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:26.752587 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:26.753906 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:26.754173 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:26.754913 (+ 740us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 20:25:26.775667 (+ 20754us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 20:25:26.776456 (+ 789us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60890: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":100}
W20260806 20:25:26.777599 23448 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60890: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260806 20:25:26.781842 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 23375
I20260806 20:25:26.784171 23440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:26.913352 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 23375
2026-08-06T20:25:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (1261 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 20:25:26.986302 17847 mini_oidc.cc:152] Starting JWKS server
I20260806 20:25:26.989502 17847 webserver.cc:460] Webserver started at https://127.0.0.1:39387/ using document root <none> and password file <none>
I20260806 20:25:26.989826 17847 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 20:25:26.990878 17847 webserver.cc:460] Webserver started at http://127.0.0.1:44317/ using document root <none> and password file <none>
2026-08-06T20:25:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:27Z Disabled control of system clock
I20260806 20:25:27.047210 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:33087
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:33087
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39387/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 20:25:27.628422 23482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:27.629020 23482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:27.629220 23482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:27.642709 23482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:27.642894 23482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:27.642953 23482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:27.643041 23482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:27.643111 23482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 20:25:27.643221 23482 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 20:25:27.643298 23482 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39387/jwks.json
I20260806 20:25:27.660151 23482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:33087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:33087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39387/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:27.662313 23482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:27.665874 23482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:27.687897 23490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:27.687777 23488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:27.687731 23487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:27.689313 23482 server_base.cc:1034] running on GCE node
I20260806 20:25:27.690841 23482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:27.693535 23482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:27.695084 23482 hybrid_clock.cc:648] HybridClock initialized: now 1786047927694950 us; error 107 us; skew 500 ppm
I20260806 20:25:27.695969 23482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:27.700372 23482 webserver.cc:460] Webserver started at http://127.17.109.254:35423/ using document root <none> and password file <none>
I20260806 20:25:27.701861 23482 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:27.702101 23482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:27.702693 23482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:27.708559 23482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "0f53dd508cf5409f91d50ad645f06e90"
format_stamp: "Formatted at 2026-08-06 20:25:27 on dist-test-slave-6k22"
server_key: "5aff51ebe7c3fff4c232318b066a163b"
server_key_iv: "be8dc0ea6021f9a4469c746477ebca81"
server_key_version: "encryptionkey@0"
I20260806 20:25:27.710106 23482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "0f53dd508cf5409f91d50ad645f06e90"
format_stamp: "Formatted at 2026-08-06 20:25:27 on dist-test-slave-6k22"
server_key: "5aff51ebe7c3fff4c232318b066a163b"
server_key_iv: "be8dc0ea6021f9a4469c746477ebca81"
server_key_version: "encryptionkey@0"
I20260806 20:25:27.720008 23482 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.011s sys 0.000s
I20260806 20:25:27.726533 23496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:27.729285 23482 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260806 20:25:27.729615 23482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "0f53dd508cf5409f91d50ad645f06e90"
format_stamp: "Formatted at 2026-08-06 20:25:27 on dist-test-slave-6k22"
server_key: "5aff51ebe7c3fff4c232318b066a163b"
server_key_iv: "be8dc0ea6021f9a4469c746477ebca81"
server_key_version: "encryptionkey@0"
I20260806 20:25:27.730077 23482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:27.766065 23482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:27.768042 23482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:27.768698 23482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:27.807387 23482 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:33087
I20260806 20:25:27.807396 23549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:33087 every 8 connection(s)
I20260806 20:25:27.809666 23482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:27.817703 23550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:27.818810 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 23482
I20260806 20:25:27.819211 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:27.820588 17847 external_mini_cluster.cc:1428] Setting key 70d57bc1cde9d5dee8181ba12c403c11
I20260806 20:25:27.854573 23550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90: Bootstrap starting.
I20260806 20:25:27.856065 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:27.824791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.17.109.254:33087)
0806 20:25:27.827976 (+ 3185us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:27.828038 (+ 62us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:27.828215 (+ 177us) server_negotiation.cc:407] Connection header received
0806 20:25:27.833535 (+ 5320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:27.833664 (+ 129us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:27.835151 (+ 1487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:27.836551 (+ 1400us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:27.838243 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:27.842779 (+ 4536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:27.845348 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:27.847660 (+ 2312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:27.848114 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:27.850255 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:27.850275 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:27.850363 (+ 88us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:27.851802 (+ 1439us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:27.852038 (+ 236us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:27.852100 (+ 62us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:27.852282 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:27.852706 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 20:25:27.853396 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1666,"thread_start_us":1029,"threads_started":1}
I20260806 20:25:27.860651 23550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:27.862756 23550 log.cc:824] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:27.864240 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:25:27.869179 23550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90: No bootstrap required, opened a new log
I20260806 20:25:27.881932 23550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f53dd508cf5409f91d50ad645f06e90" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33087 } }
I20260806 20:25:27.882543 23550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:27.882720 23550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f53dd508cf5409f91d50ad645f06e90, State: Initialized, Role: FOLLOWER
I20260806 20:25:27.883615 23550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f53dd508cf5409f91d50ad645f06e90" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33087 } }
I20260806 20:25:27.884081 23550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:27.884328 23550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:27.884620 23550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:27.888306 23550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f53dd508cf5409f91d50ad645f06e90" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33087 } }
I20260806 20:25:27.889184 23550 leader_election.cc:302] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f53dd508cf5409f91d50ad645f06e90; no voters:
I20260806 20:25:27.890074 23550 leader_election.cc:288] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:27.890415 23555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:27.891693 23555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [term 1 LEADER]: Becoming Leader. State: Replica: 0f53dd508cf5409f91d50ad645f06e90, State: Running, Role: LEADER
I20260806 20:25:27.892671 23555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f53dd508cf5409f91d50ad645f06e90" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33087 } }
I20260806 20:25:27.893920 23550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:27.898617 23556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f53dd508cf5409f91d50ad645f06e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f53dd508cf5409f91d50ad645f06e90" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33087 } } }
I20260806 20:25:27.898618 23557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f53dd508cf5409f91d50ad645f06e90. Latest consensus state: current_term: 1 leader_uuid: "0f53dd508cf5409f91d50ad645f06e90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f53dd508cf5409f91d50ad645f06e90" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33087 } } }
I20260806 20:25:27.899539 23556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:27.899755 23557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:27.903398 23561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:27.913902 23561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:27.931762 23561 catalog_manager.cc:1347] Generated new cluster ID: 908fee2ceda449e9b28e9df0f2715df1
I20260806 20:25:27.932045 23561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:27.956159 23561 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:27.958730 23561 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:27.970841 23561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f53dd508cf5409f91d50ad645f06e90: Generated new TSK 0
I20260806 20:25:27.972582 23561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:25:28.020606 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:28.002292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47070 (local address 127.17.109.254:33087)
0806 20:25:28.002891 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:28.002933 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:28.003915 (+ 982us) server_negotiation.cc:407] Connection header received
0806 20:25:28.004380 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:28.004384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:28.004526 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:28.004874 (+ 348us) server_negotiation.cc:226] Negotiated authn=JWT
0806 20:25:28.006606 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.007988 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.012036 (+ 4048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.012942 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.013133 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:28.014393 (+ 1260us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 20:25:28.019278 (+ 4885us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 20:25:28.020090 (+ 812us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":121}
W20260806 20:25:28.020821 23553 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 20:25:28.024483 23581 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:28.002230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:33087 (local address 127.0.0.1:47070)
0806 20:25:28.003734 (+ 1504us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:28.003772 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:28.004084 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:28.004977 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:28.005003 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:28.005109 (+ 106us) client_negotiation.cc:189] Negotiated authn=JWT
0806 20:25:28.006199 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:28.006234 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.008243 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.008260 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:28.011787 (+ 3527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:28.011824 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.013335 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.013353 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:28.013773 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:28.013880 (+ 107us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 20:25:28.019818 (+ 5938us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 20:25:28.023584 (+ 3766us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.109.254:33087: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39387/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":1199,"mutex_wait_us":2979,"thread_start_us":839,"threads_started":1}
I20260806 20:25:28.027478 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 23482
I20260806 20:25:28.029898 23545 generic_service.cc:196] Checking for leaks (request via RPC)
E20260806 20:25:28.039915 23582 webserver.cc:533] Webserver: sslize failed:
E20260806 20:25:28.040058 23582 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 20:25:28.127115 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 23482
2026-08-06T20:25:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (1200 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T20:25:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:28Z Disabled control of system clock
I20260806 20:25:28.219441 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:33555
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:33555
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/testchainca.cert with env {}
W20260806 20:25:28.553246 23587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:28.553580 23587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:28.553733 23587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:28.561273 23587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:28.561395 23587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:28.561492 23587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:28.561594 23587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:28.561681 23587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 20:25:28.561800 23587 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 20:25:28.561859 23587 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 20:25:28.572367 23587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:33555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:33555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:28.573779 23587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:28.575160 23587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:28.587826 23592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:28.587785 23595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:28.587867 23593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:28.588970 23587 server_base.cc:1034] running on GCE node
I20260806 20:25:28.589919 23587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:28.591404 23587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:28.592681 23587 hybrid_clock.cc:648] HybridClock initialized: now 1786047928592635 us; error 63 us; skew 500 ppm
I20260806 20:25:28.593081 23587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:28.595690 23587 webserver.cc:460] Webserver started at http://127.17.109.254:32851/ using document root <none> and password file <none>
I20260806 20:25:28.596428 23587 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:28.596567 23587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:28.596892 23587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:28.599926 23587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7"
format_stamp: "Formatted at 2026-08-06 20:25:28 on dist-test-slave-6k22"
server_key: "87cf11763d59077f7e8601eafaf85f8b"
server_key_iv: "414302d477d44886457f993c47a9f6e2"
server_key_version: "encryptionkey@0"
I20260806 20:25:28.600754 23587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7"
format_stamp: "Formatted at 2026-08-06 20:25:28 on dist-test-slave-6k22"
server_key: "87cf11763d59077f7e8601eafaf85f8b"
server_key_iv: "414302d477d44886457f993c47a9f6e2"
server_key_version: "encryptionkey@0"
I20260806 20:25:28.605985 23587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:25:28.609639 23601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:28.611110 23587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:25:28.611328 23587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7"
format_stamp: "Formatted at 2026-08-06 20:25:28 on dist-test-slave-6k22"
server_key: "87cf11763d59077f7e8601eafaf85f8b"
server_key_iv: "414302d477d44886457f993c47a9f6e2"
server_key_version: "encryptionkey@0"
I20260806 20:25:28.611565 23587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:28.623950 23587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:28.624785 23587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:28.625082 23587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:28.644261 23587 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:33555
I20260806 20:25:28.644285 23653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:33555 every 8 connection(s)
I20260806 20:25:28.645674 23587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:28.649591 23654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:28.651436 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 23587
I20260806 20:25:28.651721 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:28.652486 17847 external_mini_cluster.cc:1428] Setting key ade53b5c17732d5554ac2bc0d0d275a1
I20260806 20:25:28.661863 23654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7: Bootstrap starting.
I20260806 20:25:28.665714 23654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:28.666867 23654 log.cc:824] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:28.667291 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:28.654501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42124 (local address 127.17.109.254:33555)
0806 20:25:28.655715 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:28.655726 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:28.655771 (+ 45us) server_negotiation.cc:407] Connection header received
0806 20:25:28.657568 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:28.657590 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:28.658095 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:28.658490 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:28.659099 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.660689 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.662602 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.663317 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.663465 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:28.664833 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:28.664842 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:28.664855 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:25:28.665401 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:28.665490 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:25:28.665511 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:28.665608 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:28.665932 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 20:25:28.666278 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":434,"threads_started":1}
I20260806 20:25:28.670110 23654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7: No bootstrap required, opened a new log
I20260806 20:25:28.671212 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:25:28.678035 23654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33555 } }
I20260806 20:25:28.678418 23654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:28.678602 23654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aba3d2e6fbf44058fbe8c4ba200aeb7, State: Initialized, Role: FOLLOWER
I20260806 20:25:28.679214 23654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33555 } }
I20260806 20:25:28.679440 23654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:28.679653 23654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:28.679832 23654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:28.682374 23654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33555 } }
I20260806 20:25:28.682898 23654 leader_election.cc:302] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2aba3d2e6fbf44058fbe8c4ba200aeb7; no voters:
I20260806 20:25:28.683542 23654 leader_election.cc:288] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:28.683712 23660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:28.684790 23660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [term 1 LEADER]: Becoming Leader. State: Replica: 2aba3d2e6fbf44058fbe8c4ba200aeb7, State: Running, Role: LEADER
I20260806 20:25:28.685376 23660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33555 } }
I20260806 20:25:28.685863 23654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:28.688030 23662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aba3d2e6fbf44058fbe8c4ba200aeb7. Latest consensus state: current_term: 1 leader_uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33555 } } }
I20260806 20:25:28.688410 23662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:28.688912 23661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aba3d2e6fbf44058fbe8c4ba200aeb7" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33555 } } }
I20260806 20:25:28.689240 23661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:28.690032 23667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:28.696429 23667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:28.705847 23667 catalog_manager.cc:1347] Generated new cluster ID: 05e56752e38443bf9b9c4ea46b85e0d8
I20260806 20:25:28.705981 23667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:28.736150 23667 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:28.738164 23667 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:28.753144 23667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2aba3d2e6fbf44058fbe8c4ba200aeb7: Generated new TSK 0
I20260806 20:25:28.754318 23667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:25:28.839618 23658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:28.777417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.17.109.254:33555)
0806 20:25:28.777752 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:28.777759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:28.778151 (+ 392us) server_negotiation.cc:407] Connection header received
0806 20:25:28.778443 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:28.778448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:28.778539 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:28.778738 (+ 199us) server_negotiation.cc:226] Negotiated authn=JWT
0806 20:25:28.779376 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.780661 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.782911 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.783381 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.783491 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:28.784122 (+ 631us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 20:25:28.838960 (+ 54838us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 20:25:28.839315 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42134: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":89}
W20260806 20:25:28.839813 23658 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42134: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 20:25:28.840200 23685 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:28.777282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:33555 (local address 127.0.0.1:42134)
0806 20:25:28.778017 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:28.778038 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:28.778227 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:28.778767 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:28.778773 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:28.778804 (+ 31us) client_negotiation.cc:189] Negotiated authn=JWT
0806 20:25:28.779169 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:28.779182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.780866 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.780874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:28.782728 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:28.782745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:28.783594 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:28.783602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:28.783806 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:28.783864 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 20:25:28.839307 (+ 55443us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 20:25:28.839723 (+ 416us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.109.254:33555: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":391,"threads_started":1}
I20260806 20:25:28.841619 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 23587
I20260806 20:25:28.843051 23649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:28.942806 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 23587
2026-08-06T20:25:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (797 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (431 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (437 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:29 dist-test-slave-6k22 krb5kdc[23701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:29 dist-test-slave-6k22 krb5kdc[23701](info): set up 2 sockets
Aug 06 20:25:29 dist-test-slave-6k22 krb5kdc[23701](info): commencing operation
krb5kdc: starting...
W20260806 20:25:31.020689 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:31 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:25:31.259275 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38323
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:38323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:31.604729 23715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:31.605131 23715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:31.605324 23715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:31.613574 23715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:31.613749 23715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:31.613858 23715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:31.613946 23715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:31.613999 23715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:31.625360 23715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:38323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:31.626636 23715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:31.628032 23715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:31.639909 23720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:31.639993 23721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:31.640040 23723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:31.641505 23715 server_base.cc:1034] running on GCE node
I20260806 20:25:31.642529 23715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:31.644150 23715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:31.645398 23715 hybrid_clock.cc:648] HybridClock initialized: now 1786047931645376 us; error 61 us; skew 500 ppm
I20260806 20:25:31.645766 23715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:31 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047931, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:31.649010 23715 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:25:31.650676 23715 webserver.cc:460] Webserver started at http://127.17.109.254:44517/ using document root <none> and password file <none>
I20260806 20:25:31.651398 23715 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:31.651549 23715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:31.651948 23715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:31.655166 23715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed"
format_stamp: "Formatted at 2026-08-06 20:25:31 on dist-test-slave-6k22"
server_key: "5dc0bbec03342e2f7eb870fc1dcfcf05"
server_key_iv: "47654e8717153c5cd667721e186e89f2"
server_key_version: "encryptionkey@0"
I20260806 20:25:31.656189 23715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed"
format_stamp: "Formatted at 2026-08-06 20:25:31 on dist-test-slave-6k22"
server_key: "5dc0bbec03342e2f7eb870fc1dcfcf05"
server_key_iv: "47654e8717153c5cd667721e186e89f2"
server_key_version: "encryptionkey@0"
I20260806 20:25:31.661686 23715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260806 20:25:31.665534 23731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:31.666931 23715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:25:31.667116 23715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed"
format_stamp: "Formatted at 2026-08-06 20:25:31 on dist-test-slave-6k22"
server_key: "5dc0bbec03342e2f7eb870fc1dcfcf05"
server_key_iv: "47654e8717153c5cd667721e186e89f2"
server_key_version: "encryptionkey@0"
I20260806 20:25:31.667331 23715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:31.688738 23715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:31.692955 23715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:31.693255 23715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:31.712543 23715 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:38323
I20260806 20:25:31.712599 23784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:38323 every 8 connection(s)
I20260806 20:25:31.713948 23715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:31.716256 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 23715
I20260806 20:25:31.716517 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:31.717375 17847 external_mini_cluster.cc:1428] Setting key 77ea91c6291e040554925ad637e5e52f
I20260806 20:25:31.718712 23785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 20:25:31 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:31.734256 23785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed: Bootstrap starting.
I20260806 20:25:31.737548 23785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:31.738976 23785 log.cc:824] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:31.742228 23788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:31.719513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49060 (local address 127.17.109.254:38323)
0806 20:25:31.720355 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:31.720366 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:31.720405 (+ 39us) server_negotiation.cc:407] Connection header received
0806 20:25:31.722015 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:31.722036 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:31.722432 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:31.722780 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:31.724905 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:31.726480 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:31.728389 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:31.729062 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:31.729196 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:31.734423 (+ 5227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:31.734435 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:31.734457 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:31.734504 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:31.737624 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:31.738512 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:31.738519 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:31.738525 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:31.738708 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:31.739241 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:31.739245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:31.739247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:31.740036 (+ 789us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:31.740386 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:31.740701 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 20:25:31.741090 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":367,"threads_started":1}
I20260806 20:25:31.742538 23785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed: No bootstrap required, opened a new log
I20260806 20:25:31.750422 23785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38323 } }
I20260806 20:25:31.750844 23785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:31.750964 23785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42b0cdbcc0394a6a8dc6cec1bc15e0ed, State: Initialized, Role: FOLLOWER
I20260806 20:25:31.751658 23785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38323 } }
I20260806 20:25:31.751935 23785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:31.752087 23785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:31.752276 23785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:31.754590 23785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38323 } }
I20260806 20:25:31.755118 23785 leader_election.cc:302] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42b0cdbcc0394a6a8dc6cec1bc15e0ed; no voters:
I20260806 20:25:31.755843 23785 leader_election.cc:288] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:31.755970 23790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:31.756734 23790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [term 1 LEADER]: Becoming Leader. State: Replica: 42b0cdbcc0394a6a8dc6cec1bc15e0ed, State: Running, Role: LEADER
I20260806 20:25:31.757400 23790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38323 } }
I20260806 20:25:31.758126 23785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:31.760653 23792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42b0cdbcc0394a6a8dc6cec1bc15e0ed. Latest consensus state: current_term: 1 leader_uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38323 } } }
I20260806 20:25:31.760553 23791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42b0cdbcc0394a6a8dc6cec1bc15e0ed" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38323 } } }
I20260806 20:25:31.760990 23791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:31.760990 23792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:31.764655 23795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:31.770711 23795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:31.781056 23795 catalog_manager.cc:1347] Generated new cluster ID: b025226431144563a3e4b542f2fdeda7
I20260806 20:25:31.781191 23795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:31.794531 23795 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:31.795969 23795 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:31.804854 23795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42b0cdbcc0394a6a8dc6cec1bc15e0ed: Generated new TSK 0
I20260806 20:25:31.806005 23795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:25:31.874778 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38323
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:32.215027 23815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:32.215309 23815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:32.215409 23815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:32.222472 23815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:32.222572 23815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:32.222690 23815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:32.232045 23815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:32.233222 23815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:32.234476 23815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:32.249256 23823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:32.249414 23820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:32.249298 23821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:32.253260 23815 server_base.cc:1034] running on GCE node
I20260806 20:25:32.253916 23815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:32.254917 23815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:32.256166 23815 hybrid_clock.cc:648] HybridClock initialized: now 1786047932256084 us; error 48 us; skew 500 ppm
I20260806 20:25:32.256539 23815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:32 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047932, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:32.260471 23815 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:25:32.262008 23815 webserver.cc:460] Webserver started at http://127.17.109.193:44993/ using document root <none> and password file <none>
I20260806 20:25:32.262720 23815 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:32.262879 23815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:32.263260 23815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:32.266464 23815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "738a9822e6de4027a7bc845f8c2da15a"
format_stamp: "Formatted at 2026-08-06 20:25:32 on dist-test-slave-6k22"
server_key: "e0f29aedb5944761278d87b284faa38c"
server_key_iv: "d974c6b510098bf0a3782b0716114827"
server_key_version: "encryptionkey@0"
I20260806 20:25:32.267339 23815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "738a9822e6de4027a7bc845f8c2da15a"
format_stamp: "Formatted at 2026-08-06 20:25:32 on dist-test-slave-6k22"
server_key: "e0f29aedb5944761278d87b284faa38c"
server_key_iv: "d974c6b510098bf0a3782b0716114827"
server_key_version: "encryptionkey@0"
I20260806 20:25:32.273046 23815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 20:25:32.276831 23831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:32.278167 23815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:32.278398 23815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "738a9822e6de4027a7bc845f8c2da15a"
format_stamp: "Formatted at 2026-08-06 20:25:32 on dist-test-slave-6k22"
server_key: "e0f29aedb5944761278d87b284faa38c"
server_key_iv: "d974c6b510098bf0a3782b0716114827"
server_key_version: "encryptionkey@0"
I20260806 20:25:32.278589 23815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:32.295641 23815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:32.300032 23815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:32.300338 23815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:32.301358 23815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:32.302713 23815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:32.302788 23815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:32.302935 23815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:32.303004 23815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:32.337474 23815 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:39643
I20260806 20:25:32.337544 23950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:39643 every 8 connection(s)
I20260806 20:25:32.339192 23815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:32.340807 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 23815
I20260806 20:25:32.341069 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:32.341831 17847 external_mini_cluster.cc:1428] Setting key cad8b0c79fbe6d4b0da7ad98aed089a6
Aug 06 20:25:32 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047932, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:32.362076 23954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:32.342190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:42825 (local address 127.17.109.254:38323)
0806 20:25:32.342744 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:32.342753 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:32.345823 (+ 3070us) server_negotiation.cc:407] Connection header received
0806 20:25:32.345880 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:32.345884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:32.345945 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:32.346030 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:32.348313 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:32.349029 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:32.351621 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:32.352258 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:32.352388 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:32.356194 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:32.356199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:32.356202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:32.356235 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:32.358855 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:32.359719 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:32.359729 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:32.359731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:32.359863 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:32.360304 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:32.360311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:32.360318 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:32.360575 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:32.360734 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:32.361463 (+ 729us) server_negotiation.cc:299] Negotiation successful
0806 20:25:32.361695 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":254,"threads_started":1}
I20260806 20:25:32.363152 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:32.342829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38323 (local address 127.17.109.193:42825)
0806 20:25:32.343779 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:32.343838 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:32.345539 (+ 1701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:32.346359 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:32.346372 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:32.347191 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:32.348103 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:32.348133 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:32.349212 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:32.349220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:32.351455 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:32.351474 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:32.352511 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:32.352520 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:32.352847 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:32.353660 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:32.353689 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:32.355956 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:32.359000 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:32.359009 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:32.359026 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:32.359602 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:32.359978 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:32.359984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:32.359987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:32.360181 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:32.360707 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:32.360715 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:32.361058 (+ 343us) client_negotiation.cc:769] Sending connection context
0806 20:25:32.361350 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 20:25:32.361886 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":415,"threads_started":1}
I20260806 20:25:32.364835 23952 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38323
I20260806 20:25:32.365183 23952 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:32.366130 23952 heartbeater.cc:511] Master 127.17.109.254:38323 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
I20260806 20:25:32.368570 23748 ts_manager.cc:194] Registered new tserver with Master: 738a9822e6de4027a7bc845f8c2da15a (127.17.109.193:39643)
I20260806 20:25:32.370076 23748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:42825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:25:32.399137 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38323
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:32.732887 23958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:32.733166 23958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:32.733260 23958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:32.740586 23958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:32.740711 23958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:32.740828 23958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:32.750907 23958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:32.752365 23958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:32.753749 23958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:32.767318 23964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:32.767397 23968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:32.767432 23965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:32.768357 23958 server_base.cc:1034] running on GCE node
I20260806 20:25:32.768931 23958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:32.769802 23958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:32.771009 23958 hybrid_clock.cc:648] HybridClock initialized: now 1786047932770958 us; error 41 us; skew 500 ppm
I20260806 20:25:32.771364 23958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:32 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047932, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:32.774878 23958 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:25:32.776362 23958 webserver.cc:460] Webserver started at http://127.17.109.194:44963/ using document root <none> and password file <none>
I20260806 20:25:32.777009 23958 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:32.777109 23958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:32.777366 23958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:32.780284 23958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "eeda568531ba4048a869d7fcf2fb8a34"
format_stamp: "Formatted at 2026-08-06 20:25:32 on dist-test-slave-6k22"
server_key: "66ef4f7fe4321e3775547fcb3fbd4bfd"
server_key_iv: "7943136cc9cda63b149b7311fb25ddd3"
server_key_version: "encryptionkey@0"
I20260806 20:25:32.781003 23958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "eeda568531ba4048a869d7fcf2fb8a34"
format_stamp: "Formatted at 2026-08-06 20:25:32 on dist-test-slave-6k22"
server_key: "66ef4f7fe4321e3775547fcb3fbd4bfd"
server_key_iv: "7943136cc9cda63b149b7311fb25ddd3"
server_key_version: "encryptionkey@0"
I20260806 20:25:32.786396 23958 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:25:32.789757 23975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:32.791101 23958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:25:32.791266 23958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "eeda568531ba4048a869d7fcf2fb8a34"
format_stamp: "Formatted at 2026-08-06 20:25:32 on dist-test-slave-6k22"
server_key: "66ef4f7fe4321e3775547fcb3fbd4bfd"
server_key_iv: "7943136cc9cda63b149b7311fb25ddd3"
server_key_version: "encryptionkey@0"
I20260806 20:25:32.791468 23958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:32.804791 23958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:32.809508 23958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:32.809752 23958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:32.810606 23958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:32.811921 23958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:32.811993 23958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:32.812085 23958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:32.812132 23958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:32.845988 23958 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:34581
I20260806 20:25:32.846035 24094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:34581 every 8 connection(s)
I20260806 20:25:32.847280 23958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:32.854799 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 23958
I20260806 20:25:32.855033 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:32.855693 17847 external_mini_cluster.cc:1428] Setting key 4cc56555ce18341d5f7e55e1159761d7
Aug 06 20:25:32 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047932, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:32.871263 23954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:32.849912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:41093 (local address 127.17.109.254:38323)
0806 20:25:32.850146 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:32.850153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:32.851537 (+ 1384us) server_negotiation.cc:407] Connection header received
0806 20:25:32.853426 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:32.853430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:32.853513 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:32.853606 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:32.855173 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:32.856336 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:32.858069 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:32.859413 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:32.859606 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:32.863656 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:32.863666 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:32.863674 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:32.863725 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:32.867785 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:32.868641 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:32.868647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:32.868651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:32.868799 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:32.869306 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:32.869317 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:32.869319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:32.869604 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:32.869799 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:32.870546 (+ 747us) server_negotiation.cc:299] Negotiation successful
0806 20:25:32.870851 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 20:25:32.872363 24097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:32.850447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38323 (local address 127.17.109.194:41093)
0806 20:25:32.851289 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:32.851326 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:32.853145 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:32.853869 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:32.853877 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:32.854386 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:32.854932 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:32.854981 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:32.856507 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:32.856513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:32.857915 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:32.857927 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:32.859754 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:32.859760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:32.859970 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:32.860709 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:32.860755 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:32.863347 (+ 2592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:32.867960 (+ 4613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:32.867970 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:32.867991 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:32.868490 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:32.868928 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:32.868935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:32.868939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:32.869165 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:32.869771 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:32.869779 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:32.870146 (+ 367us) client_negotiation.cc:769] Sending connection context
0806 20:25:32.870441 (+ 295us) client_negotiation.cc:240] Negotiation successful
0806 20:25:32.870995 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":408,"threads_started":1}
I20260806 20:25:32.873939 24095 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38323
I20260806 20:25:32.874297 24095 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:32.875095 24095 heartbeater.cc:511] Master 127.17.109.254:38323 requested a full tablet report, sending...
I20260806 20:25:32.876691 23748 ts_manager.cc:194] Registered new tserver with Master: eeda568531ba4048a869d7fcf2fb8a34 (127.17.109.194:34581)
I20260806 20:25:32.877574 23748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:41093
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:25:32.914273 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38323
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:33.244264 24100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:33.244575 24100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:33.244762 24100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:33.252146 24100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:33.252249 24100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:33.252418 24100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:33.262240 24100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:33.263501 24100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:33.264974 24100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:33.278069 24105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:33.278138 24108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:33.278426 24106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:33.279449 24100 server_base.cc:1034] running on GCE node
I20260806 20:25:33.280092 24100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:33.281054 24100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:33.282290 24100 hybrid_clock.cc:648] HybridClock initialized: now 1786047933282252 us; error 48 us; skew 500 ppm
I20260806 20:25:33.282656 24100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:33.286099 24100 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:25:33.287689 24100 webserver.cc:460] Webserver started at http://127.17.109.195:44641/ using document root <none> and password file <none>
I20260806 20:25:33.288388 24100 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:33.288532 24100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:33.288863 24100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:33.291929 24100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "7cd9ec17e49c4628b0e215c06efdc898"
format_stamp: "Formatted at 2026-08-06 20:25:33 on dist-test-slave-6k22"
server_key: "0fd61a341f1011ac435eadc85f9d62f6"
server_key_iv: "c61e97fa11a7efe059fcc5992fbb664f"
server_key_version: "encryptionkey@0"
I20260806 20:25:33.292738 24100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "7cd9ec17e49c4628b0e215c06efdc898"
format_stamp: "Formatted at 2026-08-06 20:25:33 on dist-test-slave-6k22"
server_key: "0fd61a341f1011ac435eadc85f9d62f6"
server_key_iv: "c61e97fa11a7efe059fcc5992fbb664f"
server_key_version: "encryptionkey@0"
I20260806 20:25:33.298324 24100 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:25:33.301856 24117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:33.303385 24100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 20:25:33.303689 24100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "7cd9ec17e49c4628b0e215c06efdc898"
format_stamp: "Formatted at 2026-08-06 20:25:33 on dist-test-slave-6k22"
server_key: "0fd61a341f1011ac435eadc85f9d62f6"
server_key_iv: "c61e97fa11a7efe059fcc5992fbb664f"
server_key_version: "encryptionkey@0"
I20260806 20:25:33.303928 24100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:33.327368 24100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:33.332332 24100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:33.332671 24100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:33.333734 24100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:33.335116 24100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:33.335230 24100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:33.335388 24100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:33.335464 24100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:33.373539 23952 heartbeater.cc:503] Master 127.17.109.254:38323 was elected leader, sending a full tablet report...
I20260806 20:25:33.374476 24100 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:40383
I20260806 20:25:33.374522 24234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:40383 every 8 connection(s)
I20260806 20:25:33.375978 24100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:33.379499 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 24100
I20260806 20:25:33.379757 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:33.380219 17847 external_mini_cluster.cc:1428] Setting key 25fc301e353a3b86697487e275b748dc
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:33.399061 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:33.378777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:36561 (local address 127.17.109.254:38323)
0806 20:25:33.379259 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.379266 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.380745 (+ 1479us) server_negotiation.cc:407] Connection header received
0806 20:25:33.382965 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.382971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.383063 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.383205 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.384885 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.385786 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.387493 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.388062 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.388215 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:33.392011 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:33.392017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:33.392020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:33.392065 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:33.395517 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.396661 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.396666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.396668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.396815 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.397303 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.397308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.397310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.397594 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:33.397716 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:33.398518 (+ 802us) server_negotiation.cc:299] Negotiation successful
0806 20:25:33.398762 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":218,"threads_started":1}
I20260806 20:25:33.400164 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:33.379529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38323 (local address 127.17.109.195:36561)
0806 20:25:33.380567 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:33.380618 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:33.382645 (+ 2027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:33.383476 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:33.383486 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:33.384097 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:33.384704 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:33.384723 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.385970 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.385975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:33.387331 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:33.387344 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.388319 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.388327 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:33.388577 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:33.389304 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:33.389332 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:33.391760 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:33.395696 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:33.395718 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:33.395738 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:33.396508 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:33.396948 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:33.396952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:33.396954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:33.397162 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:33.397722 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:33.397728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:33.398074 (+ 346us) client_negotiation.cc:769] Sending connection context
0806 20:25:33.398339 (+ 265us) client_negotiation.cc:240] Negotiation successful
0806 20:25:33.398845 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":433,"threads_started":1}
I20260806 20:25:33.401670 24235 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38323
I20260806 20:25:33.402057 24235 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:33.402977 24235 heartbeater.cc:511] Master 127.17.109.254:38323 requested a full tablet report, sending...
I20260806 20:25:33.404750 23748 ts_manager.cc:194] Registered new tserver with Master: 7cd9ec17e49c4628b0e215c06efdc898 (127.17.109.195:40383)
I20260806 20:25:33.405593 23748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:36561
I20260806 20:25:33.409581 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/test-admin.keytab.
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:33.442829 17847 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 20:25:33.448846 24243 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:33.445910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48508 (local address 127.17.109.193:39643)
0806 20:25:33.446687 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.446698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.446733 (+ 35us) server_negotiation.cc:407] Connection header received
0806 20:25:33.446804 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.446811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.447032 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.447223 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.448360 (+ 1137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48508: BlockingRecv error: recv got EOF from 127.0.0.1:48508 (error 108)
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":390,"threads_started":1}
W20260806 20:25:33.450181 17847 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc)
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:33.451807 17847 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:33.453262 17847 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:25:33.469475 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:33.454819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.17.109.193:39643)
0806 20:25:33.455042 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.455048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.455333 (+ 285us) server_negotiation.cc:407] Connection header received
0806 20:25:33.455479 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.455483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.455547 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.455699 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.456650 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.457704 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.459028 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.459751 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.459852 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:33.463465 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:33.463477 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:33.463491 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:33.463527 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:33.466673 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.467304 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.467311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.467313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.467451 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.467893 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.467897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.467900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.468322 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:33.468568 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:33.468845 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 20:25:33.469187 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 20:25:33.475948 24243 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:33.474222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48534 (local address 127.17.109.193:39643)
0806 20:25:33.474425 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.474430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.474578 (+ 148us) server_negotiation.cc:407] Connection header received
0806 20:25:33.474721 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.474724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.474791 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.474870 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.475687 (+ 817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48534: BlockingRecv error: recv got EOF from 127.0.0.1:48534 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 20:25:33.476099 17847 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc' of type FILE
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:33.477839 17847 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:33.479519 17847 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:25:33.495368 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:33.481075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.17.109.193:39643)
0806 20:25:33.481344 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.481352 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.481450 (+ 98us) server_negotiation.cc:407] Connection header received
0806 20:25:33.481688 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.481693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.481765 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.481876 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.483000 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.483931 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.485264 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.485750 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.485898 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:33.489988 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:33.489994 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:33.489997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:33.490047 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:33.492777 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.493500 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.493505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.493507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.493641 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.494109 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.494113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.494115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.494409 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:33.494711 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:33.494961 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 20:25:33.495130 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 20:25:33.500325 24243 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:33.498186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48550 (local address 127.17.109.193:39643)
0806 20:25:33.498446 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.498456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.498766 (+ 310us) server_negotiation.cc:407] Connection header received
0806 20:25:33.498988 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.498994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.499075 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.499182 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.500055 (+ 873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48550: BlockingRecv error: recv got EOF from 127.0.0.1:48550 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 20:25:33.500519 17847 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786047878953513-17847-0/krb5kdc/krb5cc)
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:33.502017 17847 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:33.503297 17847 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:25:33.517539 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:33.504625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.17.109.193:39643)
0806 20:25:33.504892 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.504898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.505170 (+ 272us) server_negotiation.cc:407] Connection header received
0806 20:25:33.505229 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.505232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.505291 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.505387 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.506299 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.507062 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.508366 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.508773 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.508904 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:33.512427 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:33.512431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:33.512435 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:33.512465 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:33.514889 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.515534 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.515541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.515544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.515737 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.516109 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.516114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.516118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.516412 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:33.516726 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:33.517011 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 20:25:33.517227 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 20:25:33.519212 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 23815
I20260806 20:25:33.532912 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:33.519937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.17.109.193:39643)
0806 20:25:33.520185 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.520191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.520496 (+ 305us) server_negotiation.cc:407] Connection header received
0806 20:25:33.520717 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.520722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.520786 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.520896 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.521962 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.522698 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.524074 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.524546 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.524710 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:33.526909 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:33.526914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:33.526918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:33.526949 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:33.529621 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.530319 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.530327 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.530331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.530487 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.530881 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.530888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.530891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.531172 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:33.531470 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:33.532271 (+ 801us) server_negotiation.cc:299] Negotiation successful
0806 20:25:33.532519 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:25:33.533576 23946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:33.623385 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 23815
I20260806 20:25:33.642629 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 23958
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:25:33.659471 24258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:33.643694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56094 (local address 127.17.109.194:34581)
0806 20:25:33.644261 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.644267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.644294 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:25:33.644356 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.644362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.644568 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.644754 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.645779 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.646989 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.648441 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.649012 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.649167 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:33.653431 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:33.653442 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:33.653456 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:33.653493 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:33.656461 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.657239 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.657247 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.657249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.657374 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.657800 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.657804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.657806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.658189 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:33.658548 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:33.658884 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 20:25:33.659163 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":248,"threads_started":1}
I20260806 20:25:33.660996 24089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:33.745230 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 23958
I20260806 20:25:33.765405 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 24100
Aug 06 20:25:33 dist-test-slave-6k22 krb5kdc[23701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:25:33.781874 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:33.766503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33730 (local address 127.17.109.195:40383)
0806 20:25:33.767160 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:33.767167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:33.767197 (+ 30us) server_negotiation.cc:407] Connection header received
0806 20:25:33.767273 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:33.767280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:33.767507 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:33.767744 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:33.768926 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.769944 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.771360 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:33.772247 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:33.772387 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:33.776399 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:33.776409 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:33.776423 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:33.776461 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:33.779335 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.780061 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.780069 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.780071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.780226 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:33.780560 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:33.780563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:33.780565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:33.780914 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:33.781182 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:33.781364 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 20:25:33.781566 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":353,"threads_started":1}
I20260806 20:25:33.783231 24230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:33.866420 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 24100
I20260806 20:25:33.885177 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 23715
I20260806 20:25:33.886544 23780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:33.991887 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 23715
2026-08-06T20:25:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4186 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:34 dist-test-slave-6k22 krb5kdc[24265](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:34 dist-test-slave-6k22 krb5kdc[24265](info): set up 2 sockets
Aug 06 20:25:34 dist-test-slave-6k22 krb5kdc[24265](info): commencing operation
krb5kdc: starting...
W20260806 20:25:35.204162 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:35 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.254.keytab.
Entry for principal oryx/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.254.keytab.
I20260806 20:25:35.451313 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:40431
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.254.keytab
--principal=oryx/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:40431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:35.780304 24278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:35.780596 24278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:35.780699 24278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:35.787714 24278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:35.787820 24278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:35.787866 24278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:35.787915 24278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:35.787961 24278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:35.797590 24278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:40431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.254.keytab
--principal=oryx/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:40431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:35.798942 24278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:35.800208 24278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:35.811944 24286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:35.812007 24283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:35.811977 24284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:35.813426 24278 server_base.cc:1034] running on GCE node
I20260806 20:25:35.814211 24278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:35.815784 24278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:35.816990 24278 hybrid_clock.cc:648] HybridClock initialized: now 1786047935816948 us; error 34 us; skew 500 ppm
I20260806 20:25:35.817349 24278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:35 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:35.828456 24278 init.cc:377] Logged in from keytab as oryx/127.17.109.254@KRBTEST.COM (short username oryx)
I20260806 20:25:35.830156 24278 webserver.cc:460] Webserver started at http://127.17.109.254:41523/ using document root <none> and password file <none>
I20260806 20:25:35.830888 24278 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:35.831015 24278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:35.831341 24278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:35.834492 24278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5"
format_stamp: "Formatted at 2026-08-06 20:25:35 on dist-test-slave-6k22"
server_key: "9623dd82bef64c7b5c98033e1f171d11"
server_key_iv: "688bc1780813d9f6e252d2b5b4b1a0ed"
server_key_version: "encryptionkey@0"
I20260806 20:25:35.835291 24278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5"
format_stamp: "Formatted at 2026-08-06 20:25:35 on dist-test-slave-6k22"
server_key: "9623dd82bef64c7b5c98033e1f171d11"
server_key_iv: "688bc1780813d9f6e252d2b5b4b1a0ed"
server_key_version: "encryptionkey@0"
I20260806 20:25:35.841100 24278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260806 20:25:35.844904 24293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:35.846416 24278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 20:25:35.846624 24278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5"
format_stamp: "Formatted at 2026-08-06 20:25:35 on dist-test-slave-6k22"
server_key: "9623dd82bef64c7b5c98033e1f171d11"
server_key_iv: "688bc1780813d9f6e252d2b5b4b1a0ed"
server_key_version: "encryptionkey@0"
I20260806 20:25:35.846892 24278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:35.866921 24278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:35.871310 24278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:35.871651 24278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:35.890796 24278 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:40431
I20260806 20:25:35.890839 24346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:40431 every 8 connection(s)
I20260806 20:25:35.892324 24278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:35.896210 24347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:35.896443 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 24278
I20260806 20:25:35.896663 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:35.897320 17847 external_mini_cluster.cc:1428] Setting key bc09f7a894dc665176b22914353d373b
I20260806 20:25:35.911942 24347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: Bootstrap starting.
Aug 06 20:25:35 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:35.915868 24347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:35.917452 24347 log.cc:824] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:35.921207 24347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: No bootstrap required, opened a new log
I20260806 20:25:35.924468 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:35.899423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43008 (local address 127.17.109.254:40431)
0806 20:25:35.900355 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:35.900370 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:35.900434 (+ 64us) server_negotiation.cc:407] Connection header received
0806 20:25:35.903584 (+ 3150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:35.903614 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:35.904154 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:35.904635 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:35.905926 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:35.908196 (+ 2270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:35.909459 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:35.910580 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:35.910772 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:35.915101 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:35.915112 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:35.915132 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:35.915189 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:35.919359 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:35.920224 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:35.920233 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:35.920244 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:35.920465 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:35.920908 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:35.920913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:35.920918 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:35.921650 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:35.922000 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:35.922341 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 20:25:35.922797 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":351,"threads_started":1}
I20260806 20:25:35.930575 24347 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } }
I20260806 20:25:35.930909 24347 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:35.930984 24347 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0eb02a82a7f41d4b87f3d54a1c243d5, State: Initialized, Role: FOLLOWER
I20260806 20:25:35.931459 24347 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } }
I20260806 20:25:35.931728 24347 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:35.931834 24347 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:35.931953 24347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:35.934369 24347 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } }
I20260806 20:25:35.934809 24347 leader_election.cc:302] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0eb02a82a7f41d4b87f3d54a1c243d5; no voters:
I20260806 20:25:35.935362 24347 leader_election.cc:288] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:35.935523 24352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:35.936276 24352 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 1 LEADER]: Becoming Leader. State: Replica: f0eb02a82a7f41d4b87f3d54a1c243d5, State: Running, Role: LEADER
I20260806 20:25:35.936801 24352 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } }
I20260806 20:25:35.937495 24347 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:35.939845 24354 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0eb02a82a7f41d4b87f3d54a1c243d5. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } }
I20260806 20:25:35.940219 24354 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:35.941780 24353 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } }
I20260806 20:25:35.942183 24353 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:35.944506 24362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:35.949954 24362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:35.960017 24362 catalog_manager.cc:1347] Generated new cluster ID: f7cb91e23f7e4e45b613e3176eff6d62
I20260806 20:25:35.960156 24362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:36.008370 24362 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:36.010515 24362 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:36.032480 24362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: Generated new TSK 0
I20260806 20:25:36.033713 24362 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab.
Entry for principal oryx/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab.
I20260806 20:25:36.103091 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40431
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab
--principal=oryx/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:36.466498 24374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:36.466811 24374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:36.467000 24374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:36.475075 24374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:36.475204 24374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:36.475386 24374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:36.485514 24374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab
--principal=oryx/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:36.486995 24374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:36.488430 24374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:36.501871 24383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:36.501989 24379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:36.501981 24380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:36.502846 24374 server_base.cc:1034] running on GCE node
I20260806 20:25:36.503450 24374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:36.504485 24374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:36.505722 24374 hybrid_clock.cc:648] HybridClock initialized: now 1786047936505662 us; error 47 us; skew 500 ppm
I20260806 20:25:36.506049 24374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:36 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047936, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:36.509397 24374 init.cc:377] Logged in from keytab as oryx/127.17.109.193@KRBTEST.COM (short username oryx)
I20260806 20:25:36.510927 24374 webserver.cc:460] Webserver started at http://127.17.109.193:39973/ using document root <none> and password file <none>
I20260806 20:25:36.511639 24374 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:36.511751 24374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:36.512027 24374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:36.514936 24374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "eda73047fa6741b3b9fe04c0c6eb62fa"
format_stamp: "Formatted at 2026-08-06 20:25:36 on dist-test-slave-6k22"
server_key: "0aea1f78740bb13cd4bd5ba9fc63d201"
server_key_iv: "23511ab326e56d40f8561c16f244ac00"
server_key_version: "encryptionkey@0"
I20260806 20:25:36.515786 24374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "eda73047fa6741b3b9fe04c0c6eb62fa"
format_stamp: "Formatted at 2026-08-06 20:25:36 on dist-test-slave-6k22"
server_key: "0aea1f78740bb13cd4bd5ba9fc63d201"
server_key_iv: "23511ab326e56d40f8561c16f244ac00"
server_key_version: "encryptionkey@0"
I20260806 20:25:36.521090 24374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 20:25:36.524576 24391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:36.526064 24374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:25:36.526207 24374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "eda73047fa6741b3b9fe04c0c6eb62fa"
format_stamp: "Formatted at 2026-08-06 20:25:36 on dist-test-slave-6k22"
server_key: "0aea1f78740bb13cd4bd5ba9fc63d201"
server_key_iv: "23511ab326e56d40f8561c16f244ac00"
server_key_version: "encryptionkey@0"
I20260806 20:25:36.526377 24374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:36.542557 24374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:36.547247 24374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:36.547511 24374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:36.548677 24374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:36.550288 24374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:36.550395 24374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:36.550559 24374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:36.550660 24374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:36.588415 24374 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:38705
I20260806 20:25:36.588466 24511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:38705 every 8 connection(s)
I20260806 20:25:36.589847 24374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:36.599335 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 24374
I20260806 20:25:36.599603 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:36.600313 17847 external_mini_cluster.cc:1428] Setting key 20c035525e219b16fe977183d649f82b
Aug 06 20:25:36 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047936, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:36.612987 24514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:36.592558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:38969 (local address 127.17.109.254:40431)
0806 20:25:36.593001 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:36.593006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:36.594033 (+ 1027us) server_negotiation.cc:407] Connection header received
0806 20:25:36.596159 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:36.596165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:36.596274 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:36.596407 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:36.597998 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:36.598884 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:36.600793 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:36.601400 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:36.601536 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:36.605165 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:36.605175 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:36.605184 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:36.605237 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:36.609366 (+ 4129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:36.610094 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:36.610103 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:36.610110 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:36.610291 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:36.610732 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:36.610740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:36.610747 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:36.611190 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:36.611359 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:36.612153 (+ 794us) server_negotiation.cc:299] Negotiation successful
0806 20:25:36.612460 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":216,"threads_started":1}
I20260806 20:25:36.613631 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:36.593114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.193:38969)
0806 20:25:36.593865 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:36.593906 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:36.595863 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:36.596663 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:36.596672 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:36.597225 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:36.597815 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:36.597834 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:36.599085 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:36.599089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:36.600635 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:36.600647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:36.601641 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:36.601648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:36.601943 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:36.602746 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:36.602768 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:36.604921 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:36.609516 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:36.609522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:36.609535 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:36.609949 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:36.610408 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:36.610412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:36.610414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:36.610606 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:36.611320 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:36.611326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:36.611723 (+ 397us) client_negotiation.cc:769] Sending connection context
0806 20:25:36.611966 (+ 243us) client_negotiation.cc:240] Negotiation successful
0806 20:25:36.612376 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":285,"threads_started":1}
I20260806 20:25:36.615360 24512 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40431
I20260806 20:25:36.615758 24512 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:36.616564 24512 heartbeater.cc:511] Master 127.17.109.254:40431 requested a full tablet report, sending...
I20260806 20:25:36.618795 24311 ts_manager.cc:194] Registered new tserver with Master: eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705)
I20260806 20:25:36.620378 24311 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:38969
WARNING: no policy specified for oryx/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab.
Entry for principal oryx/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab.
I20260806 20:25:36.662570 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40431
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab
--principal=oryx/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:36.995544 24518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:36.995947 24518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:36.996145 24518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:37.003505 24518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:37.003726 24518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:37.003949 24518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:37.014668 24518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab
--principal=oryx/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:37.016152 24518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:37.017643 24518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:37.032068 24523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:37.032099 24524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:37.032374 24526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:37.033154 24518 server_base.cc:1034] running on GCE node
I20260806 20:25:37.033754 24518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:37.034680 24518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:37.035880 24518 hybrid_clock.cc:648] HybridClock initialized: now 1786047937035838 us; error 33 us; skew 500 ppm
I20260806 20:25:37.036151 24518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:37 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047937, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:37.039754 24518 init.cc:377] Logged in from keytab as oryx/127.17.109.194@KRBTEST.COM (short username oryx)
I20260806 20:25:37.041190 24518 webserver.cc:460] Webserver started at http://127.17.109.194:44017/ using document root <none> and password file <none>
I20260806 20:25:37.041884 24518 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:37.042016 24518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:37.042346 24518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:37.045495 24518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "a641597171fd4befae1ecec6faaf9dbc"
format_stamp: "Formatted at 2026-08-06 20:25:37 on dist-test-slave-6k22"
server_key: "0aee0f56da2414e862984d87c591308d"
server_key_iv: "ed3004dc669a23b113b141c00df58d62"
server_key_version: "encryptionkey@0"
I20260806 20:25:37.046303 24518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "a641597171fd4befae1ecec6faaf9dbc"
format_stamp: "Formatted at 2026-08-06 20:25:37 on dist-test-slave-6k22"
server_key: "0aee0f56da2414e862984d87c591308d"
server_key_iv: "ed3004dc669a23b113b141c00df58d62"
server_key_version: "encryptionkey@0"
I20260806 20:25:37.051869 24518 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:25:37.055511 24533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:37.057004 24518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:37.057178 24518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "a641597171fd4befae1ecec6faaf9dbc"
format_stamp: "Formatted at 2026-08-06 20:25:37 on dist-test-slave-6k22"
server_key: "0aee0f56da2414e862984d87c591308d"
server_key_iv: "ed3004dc669a23b113b141c00df58d62"
server_key_version: "encryptionkey@0"
I20260806 20:25:37.057400 24518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:37.069646 24518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:37.074054 24518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:37.074380 24518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:37.075525 24518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:37.076992 24518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:37.077098 24518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:37.077252 24518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:37.077332 24518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:37.113430 24518 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:46033
I20260806 20:25:37.113543 24651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:46033 every 8 connection(s)
I20260806 20:25:37.114897 24518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:37.118512 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 24518
I20260806 20:25:37.118777 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:37.119349 17847 external_mini_cluster.cc:1428] Setting key 20c4257cf00e3ec248b267adefbb1aa7
Aug 06 20:25:37 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047937, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:37.137598 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:37.117730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:42483 (local address 127.17.109.254:40431)
0806 20:25:37.118272 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:37.118277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:37.119335 (+ 1058us) server_negotiation.cc:407] Connection header received
0806 20:25:37.121601 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:37.121606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:37.121720 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:37.121844 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:37.123483 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:37.124339 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:37.126015 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:37.126541 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:37.126698 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:37.130512 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:37.130517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:37.130520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:37.130551 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:37.134207 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:37.134947 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:37.134955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:37.134962 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:37.135129 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:37.135541 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:37.135548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:37.135586 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:37.136012 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:37.136178 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:37.136826 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 20:25:37.137076 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":308,"threads_started":1}
I20260806 20:25:37.138420 24655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:37.118195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.194:42483)
0806 20:25:37.119157 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:37.119210 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:37.121336 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:37.122088 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:37.122097 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:37.122704 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:37.123306 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:37.123324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:37.124493 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:37.124501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:37.125860 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:37.125872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:37.126781 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:37.126790 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:37.127046 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:37.127840 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:37.127863 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:37.130279 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:37.134356 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:37.134362 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:37.134374 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:37.134823 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:37.135253 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:37.135263 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:37.135265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:37.135433 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:37.136142 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:37.136148 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:37.136466 (+ 318us) client_negotiation.cc:769] Sending connection context
0806 20:25:37.136698 (+ 232us) client_negotiation.cc:240] Negotiation successful
0806 20:25:37.137084 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":412,"threads_started":1}
I20260806 20:25:37.140024 24652 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40431
I20260806 20:25:37.140408 24652 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:37.141292 24652 heartbeater.cc:511] Master 127.17.109.254:40431 requested a full tablet report, sending...
I20260806 20:25:37.142980 24311 ts_manager.cc:194] Registered new tserver with Master: a641597171fd4befae1ecec6faaf9dbc (127.17.109.194:46033)
I20260806 20:25:37.143931 24311 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:42483
WARNING: no policy specified for oryx/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab.
Entry for principal oryx/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab.
I20260806 20:25:37.180101 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40431
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab
--principal=oryx/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:37.508236 24658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:37.508500 24658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:37.508592 24658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:37.515847 24658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:37.515959 24658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:37.516072 24658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:37.525624 24658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab
--principal=oryx/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:37.526865 24658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:37.528119 24658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:37.542325 24664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:37.542294 24665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:37.542268 24667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:37.542904 24658 server_base.cc:1034] running on GCE node
I20260806 20:25:37.543623 24658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:37.544776 24658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:37.546037 24658 hybrid_clock.cc:648] HybridClock initialized: now 1786047937545954 us; error 69 us; skew 500 ppm
I20260806 20:25:37.546450 24658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:37 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047937, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:37.550207 24658 init.cc:377] Logged in from keytab as oryx/127.17.109.195@KRBTEST.COM (short username oryx)
I20260806 20:25:37.551959 24658 webserver.cc:460] Webserver started at http://127.17.109.195:45971/ using document root <none> and password file <none>
I20260806 20:25:37.552727 24658 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:37.552879 24658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:37.553247 24658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:37.556855 24658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "cf387168244743de91a29001aad33725"
format_stamp: "Formatted at 2026-08-06 20:25:37 on dist-test-slave-6k22"
server_key: "bf73ad20b473d2313203b12914f46e45"
server_key_iv: "f493dec9471c444f84968d5e7aac7158"
server_key_version: "encryptionkey@0"
I20260806 20:25:37.557731 24658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "cf387168244743de91a29001aad33725"
format_stamp: "Formatted at 2026-08-06 20:25:37 on dist-test-slave-6k22"
server_key: "bf73ad20b473d2313203b12914f46e45"
server_key_iv: "f493dec9471c444f84968d5e7aac7158"
server_key_version: "encryptionkey@0"
I20260806 20:25:37.563828 24658 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 20:25:37.567719 24674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:37.569211 24658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:37.569418 24658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "cf387168244743de91a29001aad33725"
format_stamp: "Formatted at 2026-08-06 20:25:37 on dist-test-slave-6k22"
server_key: "bf73ad20b473d2313203b12914f46e45"
server_key_iv: "f493dec9471c444f84968d5e7aac7158"
server_key_version: "encryptionkey@0"
I20260806 20:25:37.569656 24658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:37.592572 24658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:37.597010 24658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:37.597349 24658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:37.598416 24658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:37.599920 24658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:37.600036 24658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:37.600210 24658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:37.600296 24658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:37.623544 24512 heartbeater.cc:503] Master 127.17.109.254:40431 was elected leader, sending a full tablet report...
I20260806 20:25:37.637965 24658 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:45435
I20260806 20:25:37.637987 24792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:45435 every 8 connection(s)
I20260806 20:25:37.639497 24658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:37.644433 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 24658
I20260806 20:25:37.644659 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:37.645155 17847 external_mini_cluster.cc:1428] Setting key 9559870a9e59f81b18299b033ede446f
Aug 06 20:25:37 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047937, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:37.660089 24795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:37.642186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:49839 (local address 127.17.109.254:40431)
0806 20:25:37.642636 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:37.642641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:37.643794 (+ 1153us) server_negotiation.cc:407] Connection header received
0806 20:25:37.645724 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:37.645730 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:37.645828 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:37.645948 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:37.647416 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:37.648287 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:37.649809 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:37.650201 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:37.650340 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:37.653991 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:37.653996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:37.654000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:37.654042 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:37.656898 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:37.657608 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:37.657613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:37.657616 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:37.657766 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:37.658148 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:37.658152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:37.658154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:37.658455 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:37.658598 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:37.659461 (+ 863us) server_negotiation.cc:299] Negotiation successful
0806 20:25:37.659709 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":246,"threads_started":1}
I20260806 20:25:37.661238 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:37.642748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.195:49839)
0806 20:25:37.643637 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:37.643674 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:37.645447 (+ 1773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:37.646179 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:37.646187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:37.646695 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:37.647249 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:37.647265 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:37.648447 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:37.648452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:37.649653 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:37.649665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:37.650480 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:37.650486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:37.650773 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:37.651503 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:37.651530 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:37.653797 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:37.657041 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:37.657048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:37.657060 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:37.657467 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:37.657876 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:37.657879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:37.657882 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:37.658051 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:37.658596 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:37.658603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:37.659029 (+ 426us) client_negotiation.cc:769] Sending connection context
0806 20:25:37.659305 (+ 276us) client_negotiation.cc:240] Negotiation successful
0806 20:25:37.659916 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":329,"threads_started":1}
I20260806 20:25:37.662684 24793 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40431
I20260806 20:25:37.662988 24793 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:37.663733 24793 heartbeater.cc:511] Master 127.17.109.254:40431 requested a full tablet report, sending...
I20260806 20:25:37.665198 24311 ts_manager.cc:194] Registered new tserver with Master: cf387168244743de91a29001aad33725 (127.17.109.195:45435)
I20260806 20:25:37.666110 24311 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:49839
I20260806 20:25:37.673623 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.109.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.253.keytab.
Entry for principal oryx/127.17.109.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.253.keytab.
I20260806 20:25:37.730849 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.253:46779
--webserver_interface=127.17.109.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.253.keytab
--principal=oryx/127.17.109.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:40431,127.17.109.253:46779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:38.075172 24799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:38.075476 24799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:38.075721 24799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:38.083046 24799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:38.083177 24799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:38.083278 24799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:38.083382 24799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:38.083478 24799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:38.093786 24799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:40431,127.17.109.253:46779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.253.keytab
--principal=oryx/127.17.109.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.253:46779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.109.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
W20260806 20:25:38.094197 24799 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.109.254:40431,127.17.109.253:46779'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 20:25:38.095300 24799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:38.096748 24799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:38.108517 24805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:38.108469 24807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:38.108560 24804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:38.109386 24799 server_base.cc:1034] running on GCE node
I20260806 20:25:38.110059 24799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:38.111347 24799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:38.112597 24799 hybrid_clock.cc:648] HybridClock initialized: now 1786047938112531 us; error 43 us; skew 500 ppm
I20260806 20:25:38.113009 24799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047938, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:38.116287 24799 init.cc:377] Logged in from keytab as oryx/127.17.109.253@KRBTEST.COM (short username oryx)
I20260806 20:25:38.117892 24799 webserver.cc:460] Webserver started at http://127.17.109.253:45707/ using document root <none> and password file <none>
I20260806 20:25:38.118584 24799 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:38.118672 24799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:38.118978 24799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:38.121949 24799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data/instance:
uuid: "c885244b0041460db0bc5fef74135f33"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "269cd9fbabb2f947f0c7e5ca3675a5fd"
server_key_iv: "33528a1cc85f1a2a3f500bed42d62a2d"
server_key_version: "encryptionkey@0"
I20260806 20:25:38.122733 24799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal/instance:
uuid: "c885244b0041460db0bc5fef74135f33"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "269cd9fbabb2f947f0c7e5ca3675a5fd"
server_key_iv: "33528a1cc85f1a2a3f500bed42d62a2d"
server_key_version: "encryptionkey@0"
I20260806 20:25:38.128036 24799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 20:25:38.131628 24814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:38.132989 24799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:38.133159 24799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
uuid: "c885244b0041460db0bc5fef74135f33"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "269cd9fbabb2f947f0c7e5ca3675a5fd"
server_key_iv: "33528a1cc85f1a2a3f500bed42d62a2d"
server_key_version: "encryptionkey@0"
I20260806 20:25:38.133389 24799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:38.147162 24652 heartbeater.cc:503] Master 127.17.109.254:40431 was elected leader, sending a full tablet report...
I20260806 20:25:38.156028 24799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:38.160589 24799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:38.160928 24799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:38.186998 24799 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.253:46779
I20260806 20:25:38.187032 24869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.253:46779 every 8 connection(s)
I20260806 20:25:38.188928 24799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
I20260806 20:25:38.193979 24870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:38.197464 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 24799
I20260806 20:25:38.202837 24870 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.253@KRBTEST.COM
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047938, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.253@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:38.220976 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.205400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.17.109.254:40431)
0806 20:25:38.205964 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.205971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.206473 (+ 502us) server_negotiation.cc:407] Connection header received
0806 20:25:38.206641 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.206645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.206730 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.206824 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.208101 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.209080 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.211061 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.211776 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.211925 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.215271 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.215276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.215279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.215314 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.217958 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.218808 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.218814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.218818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.218947 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.219358 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.219364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.219367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.219757 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.219913 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.220459 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.220691 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":286,"threads_started":1}
I20260806 20:25:38.222113 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.205433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:43012)
0806 20:25:38.206317 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.206370 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.206522 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.206901 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.206907 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.207701 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.207949 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.207958 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.209260 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.209266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.210923 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.210935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.211997 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.212002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.212125 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.212746 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.212773 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.215069 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.218229 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.218239 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.218251 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.218689 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.219059 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.219064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.219068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.219245 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.219893 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.219901 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.220102 (+ 201us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.220363 (+ 261us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.221173 (+ 810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":316,"thread_start_us":477,"threads_started":1}
I20260806 20:25:38.222132 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.200087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55248 (local address 127.17.109.253:46779)
0806 20:25:38.200888 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.200900 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.200949 (+ 49us) server_negotiation.cc:407] Connection header received
0806 20:25:38.202587 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.202609 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.203009 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.203518 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.205025 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.207134 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.208955 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.210027 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.210219 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.214839 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.214847 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.214858 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.214891 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.217912 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.218612 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.218619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.218621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.218795 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.219200 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.219207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.219219 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.219794 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.220109 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.220450 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.221015 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":593,"thread_start_us":325,"threads_started":1}
I20260806 20:25:38.224303 24870 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047938, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.253@KRBTEST.COM for oryx/127.17.109.253@KRBTEST.COM
I20260806 20:25:38.238492 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.225203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.0.0.1:55260)
0806 20:25:38.225366 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.225381 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.225459 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.225814 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.225818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.226190 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.226354 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.226363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.227393 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.227397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.228337 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.228349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.229019 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.229023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.229163 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.229730 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.229747 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.232024 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.235237 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.235246 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.235251 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.235791 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.236240 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.236246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.236251 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.236497 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.237202 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.237209 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.237379 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.237547 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.237952 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 20:25:38.238533 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.225399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.17.109.253:46779)
0806 20:25:38.225547 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.225551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.225564 (+ 13us) server_negotiation.cc:407] Connection header received
0806 20:25:38.225599 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.225602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.225652 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.225728 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.226526 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.227226 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.228477 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.228895 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.228997 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.232245 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.232260 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.232264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.232296 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.235067 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.235931 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.235939 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.235946 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.236094 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.236665 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.236674 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.236682 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.236999 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.237241 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.237723 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.238102 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":49}
I20260806 20:25:38.244263 24870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Bootstrap starting.
I20260806 20:25:38.247916 24870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:38.249069 24870 log.cc:824] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:38.252266 24870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: No bootstrap required, opened a new log
I20260806 20:25:38.260108 24870 raft_consensus.cc:348] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } }
I20260806 20:25:38.260496 24870 raft_consensus.cc:374] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:38.260579 24870 raft_consensus.cc:729] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c885244b0041460db0bc5fef74135f33, State: Initialized, Role: FOLLOWER
I20260806 20:25:38.261240 24870 consensus_queue.cc:260] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } }
I20260806 20:25:38.262068 24877 sys_catalog.cc:455] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } } }
I20260806 20:25:38.262439 24877 sys_catalog.cc:458] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:25:38.263280 24870 sys_catalog.cc:564] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 20:25:38.272359 24888 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 20:25:38.272526 24888 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 20:25:38.276322 24799 master_runner.cc:428] Detected that this master c885244b0041460db0bc5fef74135f33 is joining an existing cluster
I20260806 20:25:38.276451 24799 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.109.253:46779
I20260806 20:25:38.307229 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.282922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43028 (local address 127.17.109.254:40431)
0806 20:25:38.283126 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.283147 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.289727 (+ 6580us) server_negotiation.cc:407] Connection header received
0806 20:25:38.290438 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.290448 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.290559 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.290648 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.292631 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.294148 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.296442 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.297164 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.297264 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.299985 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.299992 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.299997 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.300042 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.303856 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.304617 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.304623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.304627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.304783 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.305277 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.305283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.305288 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.305673 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.305861 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.306585 (+ 724us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.306843 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 20:25:38.308689 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.282410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:43028)
0806 20:25:38.289573 (+ 7163us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.289600 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.290226 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.290952 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.290960 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.291886 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.292411 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.292427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.294368 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.294377 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.296266 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.296283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.297545 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.297556 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.297752 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.298506 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.298531 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.299780 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.304007 (+ 4227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.304012 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.304016 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.304476 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.304940 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.304947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.304953 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.305133 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.305943 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.305949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.306191 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.307844 (+ 1653us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.308181 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6965,"thread_start_us":12470,"threads_started":1}
I20260806 20:25:38.310917 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.282467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55276 (local address 127.17.109.253:46779)
0806 20:25:38.282708 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.282722 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.294873 (+ 12151us) server_negotiation.cc:407] Connection header received
0806 20:25:38.295288 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.295293 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.295368 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.295428 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.296778 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.297884 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.300336 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.301411 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.301534 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.304332 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.304338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.304344 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.304379 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.307930 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.308741 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.308749 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.308755 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.308936 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.309374 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.309379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.309381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.309716 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.309819 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.310204 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.310552 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":43}
WARNING: no policy specified for oryx/127.17.109.252@KRBTEST.COM; defaulting to no policy
I20260806 20:25:38.311822 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.282854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.0.0.1:55276)
0806 20:25:38.294747 (+ 11893us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.294772 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.295145 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.295617 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.295622 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.296333 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.296639 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.296650 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.298505 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.298510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.300195 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.300207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.301797 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.301805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.302325 (+ 520us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.302993 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.303017 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.304158 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.308076 (+ 3918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.308084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.308087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.308599 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.309051 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.309057 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.309062 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.309253 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.309867 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.309869 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.309971 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.310103 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.311349 (+ 1246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11731,"thread_start_us":6905,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.252@KRBTEST.COM" created.
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.254@KRBTEST.COM for oryx/127.17.109.253@KRBTEST.COM
I20260806 20:25:38.334759 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.316482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55278 (local address 127.17.109.253:46779)
0806 20:25:38.316673 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.316679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.317496 (+ 817us) server_negotiation.cc:407] Connection header received
0806 20:25:38.317910 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.317918 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.318060 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.318174 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.319941 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.321254 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.323211 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.324154 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.324309 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.328660 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.328665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.328669 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.328702 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.331761 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.332457 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.332464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.332467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.332608 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.333183 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.333188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.333190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.333482 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.333647 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.334126 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.334404 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 20:25:38.334841 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.316569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.0.0.1:55278)
0806 20:25:38.317349 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.317398 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.317678 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.318232 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.318241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.319389 (+ 1148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.319720 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.319735 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.321423 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.321429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.323029 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.323044 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.324375 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.324382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.324624 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.325399 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.325427 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.328454 (+ 3027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.331898 (+ 3444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.331911 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.331919 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.332334 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.332761 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.332766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.332769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.332998 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.333639 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.333648 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.333839 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.334035 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.334386 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":354,"threads_started":1}
I20260806 20:25:38.337512 24311 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.109.253:46779: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c885244b0041460db0bc5fef74135f33"
member_type: NON_VOTER
last_known_addr {
host: "127.17.109.253"
port: 46779
}
attrs {
promote: true
}
}
dest_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5"
cas_config_opid_index: -1
I20260806 20:25:38.339063 24311 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } }
I20260806 20:25:38.346271 24900 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c885244b0041460db0bc5fef74135f33 (127.17.109.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } }
I20260806 20:25:38.347951 24844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:38.350975 24900 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.109.253:46779
I20260806 20:25:38.351617 24901 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } }
I20260806 20:25:38.352088 24901 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:38.352134 24799 master.cc:537] Master@127.17.109.253:46779 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.252.keytab.
Entry for principal oryx/127.17.109.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.252.keytab.
I20260806 20:25:38.357297 24844 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Refusing update from remote peer f0eb02a82a7f41d4b87f3d54a1c243d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 20:25:38.358675 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.252:42373
--webserver_interface=127.17.109.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.252.keytab
--principal=oryx/127.17.109.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:40431,127.17.109.253:46779,127.17.109.252:42373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:25:38.359624 24901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 20:25:38.363471 24298 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 20:25:38.374505 24799 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:38.374990 24799 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:38.375216 24799 tablet_replica.cc:330] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: stopping tablet replica
I20260806 20:25:38.380772 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.380222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.0.0.1:55288)
0806 20:25:38.380430 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.380575 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:46779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 20:25:38.381366 24298 consensus_peers.cc:487] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 -> Peer c885244b0041460db0bc5fef74135f33 (127.17.109.253:46779): Couldn't send request to peer c885244b0041460db0bc5fef74135f33. Status: Network error: Client connection negotiation failed: client connection to 127.17.109.253:46779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 20:25:38.382094 24799 master.cc:552] Master@127.17.109.253:46779 shutdown complete.
I20260806 20:25:38.382234 24799 master_runner.cc:305] Clearing existing system tablet
I20260806 20:25:38.382668 24799 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:38.382783 24799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:38.385174 24799 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 20:25:38.386755 24906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:38.387054 24799 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:38.387343 24799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 20:25:38.387435 24799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
uuid: "c885244b0041460db0bc5fef74135f33"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "269cd9fbabb2f947f0c7e5ca3675a5fd"
server_key_iv: "33528a1cc85f1a2a3f500bed42d62a2d"
server_key_version: "encryptionkey@0"
I20260806 20:25:38.391290 24799 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:38.396318 24799 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 20:25:38.396530 24799 log.cc:1196] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 20:25:38.397006 24799 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Deleting consensus metadata
I20260806 20:25:38.397274 24799 master_runner.cc:315] Copying system tablet from 127.17.109.254:40431
I20260806 20:25:38.400326 24799 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: tablet copy: Beginning tablet copy session from remote peer at address 127.17.109.254:40431
I20260806 20:25:38.415634 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.401314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43030 (local address 127.17.109.254:40431)
0806 20:25:38.401536 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.401546 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.404160 (+ 2614us) server_negotiation.cc:407] Connection header received
0806 20:25:38.404388 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.404394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.404474 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.404586 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.405405 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.406154 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.407466 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.408007 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.408145 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.409965 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.409971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.409976 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.410017 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.412973 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.413610 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.413617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.413621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.413757 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.414123 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.414129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.414132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.414426 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.414585 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.415010 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.415215 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:25:38.415668 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.401291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:43030)
0806 20:25:38.404032 (+ 2741us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.404054 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.404230 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.404613 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.404617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.405032 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.405249 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.405258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.406312 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.406317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.407306 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.407318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.408187 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.408193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.408383 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.409037 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.409055 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.409763 (+ 708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.413105 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.413112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.413116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.413482 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.413871 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.413876 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.413879 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.414013 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.414547 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.414552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.414732 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.414888 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.415204 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2601,"thread_start_us":2505,"threads_started":1}
I20260806 20:25:38.416960 24331 tablet_copy_service.cc:140] P f0eb02a82a7f41d4b87f3d54a1c243d5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c885244b0041460db0bc5fef74135f33 ({username='oryx', principal='oryx/127.17.109.253@KRBTEST.COM'} at 127.0.0.1:43030)
I20260806 20:25:38.417326 24331 tablet_copy_service.cc:161] P f0eb02a82a7f41d4b87f3d54a1c243d5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c885244b0041460db0bc5fef74135f33 at {username='oryx', principal='oryx/127.17.109.253@KRBTEST.COM'} at 127.0.0.1:43030: session id = c885244b0041460db0bc5fef74135f33-00000000000000000000000000000000
I20260806 20:25:38.420818 24331 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 20:25:38.423043 24799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 20:25:38.427917 24799 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: tablet copy: Starting download of 0 data blocks...
I20260806 20:25:38.428339 24799 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: tablet copy: Starting download of 1 WAL segments...
I20260806 20:25:38.431948 24799 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 20:25:38.435441 24331 tablet_copy_service.cc:342] P f0eb02a82a7f41d4b87f3d54a1c243d5: Request end of tablet copy session c885244b0041460db0bc5fef74135f33-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.109.253@KRBTEST.COM'} at 127.0.0.1:43030
I20260806 20:25:38.435742 24331 tablet_copy_service.cc:433] P f0eb02a82a7f41d4b87f3d54a1c243d5: ending tablet copy session c885244b0041460db0bc5fef74135f33-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c885244b0041460db0bc5fef74135f33
I20260806 20:25:38.447160 24799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:38.453025 24914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:38.457388 24915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:38.470284 24799 server_base.cc:1034] running on GCE node
W20260806 20:25:38.471913 24917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:38.472458 24799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:38.473280 24799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:38.474514 24799 hybrid_clock.cc:648] HybridClock initialized: now 1786047938474436 us; error 44 us; skew 500 ppm
I20260806 20:25:38.474859 24799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047938, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:38.477216 24799 init.cc:377] Logged in from keytab as oryx/127.17.109.253@KRBTEST.COM (short username oryx)
I20260806 20:25:38.478693 24799 webserver.cc:460] Webserver started at http://127.17.109.253:35389/ using document root <none> and password file <none>
I20260806 20:25:38.479075 24799 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:38.479184 24799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:38.482075 24799 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 20:25:38.484054 24924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:38.484532 24799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 20:25:38.484714 24799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
uuid: "c885244b0041460db0bc5fef74135f33"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "269cd9fbabb2f947f0c7e5ca3675a5fd"
server_key_iv: "33528a1cc85f1a2a3f500bed42d62a2d"
server_key_version: "encryptionkey@0"
I20260806 20:25:38.484915 24799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:38.500798 24799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:38.504925 24799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:38.527372 24799 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.253:46779
I20260806 20:25:38.527390 24976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.253:46779 every 8 connection(s)
I20260806 20:25:38.528504 24799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
I20260806 20:25:38.536547 24977 sys_catalog.cc:263] Verifying existing consensus state
I20260806 20:25:38.542709 24977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Bootstrap starting.
I20260806 20:25:38.558797 24977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:25:38.559409 24977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Bootstrap complete.
I20260806 20:25:38.560571 24977 raft_consensus.cc:348] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } }
I20260806 20:25:38.560838 24977 raft_consensus.cc:729] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c885244b0041460db0bc5fef74135f33, State: Initialized, Role: LEARNER
I20260806 20:25:38.561213 24977 consensus_queue.cc:260] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } }
I20260806 20:25:38.562367 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } }
I20260806 20:25:38.562742 24977 sys_catalog.cc:564] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:38.562903 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: This master's current role is: LEARNER
I20260806 20:25:38.576054 24991 catalog_manager.cc:1259] Loaded cluster ID: f7cb91e23f7e4e45b613e3176eff6d62
I20260806 20:25:38.576200 24991 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: loading cluster ID for follower catalog manager: success
I20260806 20:25:38.578687 24991 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: acquiring CA information for follower catalog manager: success
I20260806 20:25:38.580848 24991 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 20:25:38.668761 24793 heartbeater.cc:503] Master 127.17.109.254:40431 was elected leader, sending a full tablet report...
W20260806 20:25:38.806393 24904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:38.806773 24904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:38.806949 24904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:38.815081 24904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:38.815236 24904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:38.815354 24904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:38.815475 24904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:38.815596 24904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:38.823932 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.817133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59040 (local address 127.17.109.253:46779)
0806 20:25:38.817720 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.817725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.817743 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:25:38.817804 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.817807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.817871 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.817975 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:38.818504 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.819518 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.821929 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.822831 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.822991 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.823129 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.823500 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.823670 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":347,"threads_started":1}
I20260806 20:25:38.823932 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.816877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.0.0.1:59040)
0806 20:25:38.817140 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.817164 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.817311 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.818067 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.818073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.818114 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:38.818314 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.818327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.819693 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.819699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.821739 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.821758 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.823024 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.823031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.823284 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.823357 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.823465 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.823644 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 20:25:38.825070 24952 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260806 20:25:38.825740 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0eb02a82a7f41d4b87f3d54a1c243d5. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } }
I20260806 20:25:38.826033 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: This master's current role is: LEARNER
I20260806 20:25:38.828125 24904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:40431,127.17.109.253:46779,127.17.109.252:42373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.252.keytab
--principal=oryx/127.17.109.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.252:42373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.109.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:38.829509 24904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:38.830966 24904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:38.843259 24997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:38.843485 25000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:38.843257 24998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:38.844877 24904 server_base.cc:1034] running on GCE node
I20260806 20:25:38.845873 24904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:38.847415 24904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:38.848721 24904 hybrid_clock.cc:648] HybridClock initialized: now 1786047938848689 us; error 60 us; skew 500 ppm
I20260806 20:25:38.849110 24904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047938, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:38.852650 24904 init.cc:377] Logged in from keytab as oryx/127.17.109.252@KRBTEST.COM (short username oryx)
I20260806 20:25:38.854102 24904 webserver.cc:460] Webserver started at http://127.17.109.252:43453/ using document root <none> and password file <none>
I20260806 20:25:38.854753 24904 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:38.854856 24904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:38.855091 24904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:38.858348 24904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data/instance:
uuid: "af3861184a084cb482576e2fb6783735"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "812c326f64662350c45215186c8b48f0"
server_key_iv: "2d3dcfcb4597ca1639efea49dfb82b1e"
server_key_version: "encryptionkey@0"
I20260806 20:25:38.859092 24904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal/instance:
uuid: "af3861184a084cb482576e2fb6783735"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "812c326f64662350c45215186c8b48f0"
server_key_iv: "2d3dcfcb4597ca1639efea49dfb82b1e"
server_key_version: "encryptionkey@0"
I20260806 20:25:38.864993 24904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260806 20:25:38.868590 25007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:38.870200 24904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:38.870419 24904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
uuid: "af3861184a084cb482576e2fb6783735"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "812c326f64662350c45215186c8b48f0"
server_key_iv: "2d3dcfcb4597ca1639efea49dfb82b1e"
server_key_version: "encryptionkey@0"
I20260806 20:25:38.870606 24904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:38.885205 24904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:38.889803 24904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:38.890123 24904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:38.910562 24904 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.252:42373
I20260806 20:25:38.910598 25062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.252:42373 every 8 connection(s)
I20260806 20:25:38.912278 24904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
I20260806 20:25:38.916090 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:38.918982 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 24904
I20260806 20:25:38.923988 25063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.252@KRBTEST.COM
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047938, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.252@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:38.944865 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.927189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34280 (local address 127.17.109.254:40431)
0806 20:25:38.927740 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.927747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.928245 (+ 498us) server_negotiation.cc:407] Connection header received
0806 20:25:38.928422 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.928425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.928496 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.928796 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.929918 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.930748 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.932774 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.933513 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.933684 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.937718 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.937726 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.937733 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.937779 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.941294 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.942301 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.942307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.942310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.942468 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.942970 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.942975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.942978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.943349 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.943493 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.944256 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.944462 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":238,"threads_started":1}
I20260806 20:25:38.945652 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.921354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57640 (local address 127.17.109.252:42373)
0806 20:25:38.922137 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.922161 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.922211 (+ 50us) server_negotiation.cc:407] Connection header received
0806 20:25:38.924420 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.924450 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.924970 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.925416 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.926834 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.928818 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.930250 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.931109 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.931415 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.937010 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.937022 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.937033 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.937068 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.940902 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.941695 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.941701 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.941703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.941868 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.942324 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.942328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.942331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.943119 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.943412 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.943888 (+ 476us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.944365 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":324,"threads_started":1}
I20260806 20:25:38.945801 25068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.927178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34280)
0806 20:25:38.928096 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.928146 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.928303 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.928874 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.928883 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.929527 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.929768 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.929776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.931049 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.931055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.932575 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.932587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.933783 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.933788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.933907 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.934522 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.934543 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.937484 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.941441 (+ 3957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.941453 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.941466 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.942139 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.942592 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.942595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.942598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.942818 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.943494 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.943503 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.943866 (+ 363us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.944112 (+ 246us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.944557 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":50,"thread_start_us":503,"threads_started":1}
I20260806 20:25:38.947539 25063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047938, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.252@KRBTEST.COM for oryx/127.17.109.253@KRBTEST.COM
I20260806 20:25:38.960956 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.948381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.17.109.253:46779)
0806 20:25:38.948597 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.948604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.948711 (+ 107us) server_negotiation.cc:407] Connection header received
0806 20:25:38.948790 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.948795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.948876 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.948987 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.949645 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.950420 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.951856 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.952264 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.952357 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.955481 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.955486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.955489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.955519 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.958301 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.959084 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.959087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.959089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.959214 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.959627 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.959633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.959636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.959943 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.960088 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.960514 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.960691 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:25:38.960987 25068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.948255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.0.0.1:59054)
0806 20:25:38.948465 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.948487 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.948590 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.949039 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.949043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.949382 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.949523 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.949531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.950587 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.950591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.951694 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.951725 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.952393 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.952398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.952563 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.953095 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.953115 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.955277 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.958454 (+ 3177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.958459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.958463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.958975 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.959323 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.959326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.959328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.959467 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.960079 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.960083 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.960252 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.960454 (+ 202us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.960727 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 20:25:38.962122 25063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:25:38 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047938, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.252@KRBTEST.COM for oryx/127.17.109.252@KRBTEST.COM
I20260806 20:25:38.975033 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.963168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57642 (local address 127.17.109.252:42373)
0806 20:25:38.963326 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:38.963333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:38.963357 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:25:38.963411 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:38.963415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:38.963482 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:38.963607 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:38.964359 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.965050 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.966159 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.966546 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.966675 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.969813 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:38.969818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:38.969821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:38.969849 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:38.972549 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.973181 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.973185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.973187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.973304 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:38.973640 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:38.973643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:38.973645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:38.974011 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:38.974119 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:38.974478 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 20:25:38.974731 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":50}
I20260806 20:25:38.975040 25068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:38.962975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:42373 (local address 127.0.0.1:57642)
0806 20:25:38.963120 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:38.963136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:38.963248 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:38.963616 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:38.963620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:38.964086 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:38.964230 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.964238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.965171 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.965178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.966023 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:38.966034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:38.966684 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:38.966690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:38.966986 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:38.967551 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:38.967626 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:38.969642 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:38.972673 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.972676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.972679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.973064 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.973399 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:38.973402 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:38.973404 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:38.973551 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:38.974137 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:38.974145 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:38.974266 (+ 121us) client_negotiation.cc:769] Sending connection context
0806 20:25:38.974395 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 20:25:38.974591 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":60}
I20260806 20:25:38.980341 25063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Bootstrap starting.
I20260806 20:25:38.983501 25063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:38.984566 25063 log.cc:824] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:38.987543 25063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: No bootstrap required, opened a new log
I20260806 20:25:38.994544 25063 raft_consensus.cc:348] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } }
I20260806 20:25:38.994846 25063 raft_consensus.cc:374] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:38.994928 25063 raft_consensus.cc:729] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3861184a084cb482576e2fb6783735, State: Initialized, Role: FOLLOWER
I20260806 20:25:38.995504 25063 consensus_queue.cc:260] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } }
I20260806 20:25:38.996464 25070 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } } }
I20260806 20:25:38.996891 25070 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:25:38.997478 25063 sys_catalog.cc:564] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 20:25:39.007800 25083 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 20:25:39.007973 25083 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 20:25:39.012501 24904 master_runner.cc:428] Detected that this master af3861184a084cb482576e2fb6783735 is joining an existing cluster
I20260806 20:25:39.012742 24904 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.109.252:42373
I20260806 20:25:39.043406 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.017983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.17.109.254:40431)
0806 20:25:39.018194 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.018201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.025412 (+ 7211us) server_negotiation.cc:407] Connection header received
0806 20:25:39.025813 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.025822 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.025927 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.026006 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.028082 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.029255 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.031351 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.032010 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.032107 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.035079 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.035086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.035090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.035130 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.040082 (+ 4952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.040885 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.040893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.040899 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.041051 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.041610 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.041615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.041619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.042005 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.042124 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.042781 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.043033 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:25:39.043598 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.017870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57644 (local address 127.17.109.252:42373)
0806 20:25:39.018491 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.018501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.026807 (+ 8306us) server_negotiation.cc:407] Connection header received
0806 20:25:39.027062 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.027068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.027146 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.027213 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.028675 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.030086 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.032209 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.032999 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.033122 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.035551 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.035584 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.035591 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.035623 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.040577 (+ 4954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.041329 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.041334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.041338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.041466 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.041894 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.041898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.041902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.042269 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.042403 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.042904 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.043200 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":54}
I20260806 20:25:39.044795 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.017583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:42373 (local address 127.0.0.1:57644)
0806 20:25:39.026687 (+ 9104us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.026708 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.026917 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.027395 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.027399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.028164 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:39.028518 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.028532 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.030329 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.030340 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.032079 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.032092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.033433 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.033439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.033616 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.034254 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:39.034275 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:39.035388 (+ 1113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:39.040713 (+ 5325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.040730 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.040733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.041207 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.041608 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.041612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.041615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.041782 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.042469 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:39.042473 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:39.042621 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.043859 (+ 1238us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.044280 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8895,"thread_start_us":6934,"threads_started":1}
I20260806 20:25:39.045310 25090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.017513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34294)
0806 20:25:39.025261 (+ 7748us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.025285 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.025604 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.026293 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.026302 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.027380 (+ 1078us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:39.027895 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.027914 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.029465 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.029471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.031180 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.031195 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.032411 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.032417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.032676 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.033457 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:39.033480 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:39.034863 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:39.040252 (+ 5389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.040259 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.040264 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.040744 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.041207 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.041212 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.041231 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.041449 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.042240 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:39.042245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:39.042469 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.044158 (+ 1689us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.044471 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7429,"mutex_wait_us":9,"thread_start_us":8197,"threads_started":1}
I20260806 20:25:39.046388 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.017584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.17.109.253:46779)
0806 20:25:39.017790 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.017795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.027242 (+ 9447us) server_negotiation.cc:407] Connection header received
0806 20:25:39.027580 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.027587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.027680 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.027760 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.030896 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.032128 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.034527 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.035247 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.035357 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.038784 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.038793 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.038799 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.038844 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.042877 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.043864 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.043871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.043875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.044052 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.044595 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.044601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.044604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.044972 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.045106 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.045786 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.046044 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 20:25:39.047595 25091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.026049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.0.0.1:59060)
0806 20:25:39.027106 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.027127 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.027362 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.028007 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.028016 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.028823 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:39.030667 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.030687 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.032360 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.032368 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.034330 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.034350 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.035717 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.035725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.035957 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.036674 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:39.036707 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:39.038538 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:39.043049 (+ 4511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.043055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.043059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.043700 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.044234 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.044239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.044244 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.044442 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.045222 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:39.045229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:39.045446 (+ 217us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.046593 (+ 1147us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.046925 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":2,"thread_start_us":2829,"threads_started":1}
I20260806 20:25:39.052886 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 24374
Aug 06 20:25:39 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.254@KRBTEST.COM for oryx/127.17.109.252@KRBTEST.COM
Aug 06 20:25:39 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.193@KRBTEST.COM
I20260806 20:25:39.068922 24897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.051488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:42373 (local address 127.0.0.1:57654)
0806 20:25:39.051727 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.051748 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.051864 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.052551 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.052559 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.053085 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:39.053256 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.053265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.054446 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.054451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.055768 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.055780 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.056909 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.056917 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.057112 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.057804 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:39.057822 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:39.060822 (+ 3000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:39.065523 (+ 4701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.065531 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.065537 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.066002 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.066508 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.066515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.066519 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.066733 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.067517 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:39.067527 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:39.067832 (+ 305us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.067997 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.068344 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260806 20:25:39.070150 25066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.051577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.17.109.252:42373)
0806 20:25:39.052272 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.052276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.052289 (+ 13us) server_negotiation.cc:407] Connection header received
0806 20:25:39.052352 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.052356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.052410 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.052522 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.053395 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.054275 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.055912 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.056727 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.056875 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.061065 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.061073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.061078 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.061125 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.065338 (+ 4213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.066183 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.066197 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.066200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.066362 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.066978 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.066986 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.066994 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.067305 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.069477 (+ 2172us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.069606 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.069868 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614}
I20260806 20:25:39.071980 24311 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.109.252:42373: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "af3861184a084cb482576e2fb6783735"
member_type: NON_VOTER
last_known_addr {
host: "127.17.109.252"
port: 42373
}
attrs {
promote: true
}
}
dest_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5"
cas_config_opid_index: 5
I20260806 20:25:39.073165 24311 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } }
I20260806 20:25:39.075379 24901 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER af3861184a084cb482576e2fb6783735 (127.17.109.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.076251 24952 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 LEARNER]: Refusing update from remote peer f0eb02a82a7f41d4b87f3d54a1c243d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 20:25:39.080386 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.054355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.17.109.193:38705)
0806 20:25:39.055285 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.055297 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.055329 (+ 32us) server_negotiation.cc:407] Connection header received
0806 20:25:39.055401 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.055410 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.055746 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.055999 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.057694 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.059630 (+ 1936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.061699 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.062777 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.062955 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.069558 (+ 6603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.069578 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.069601 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.069657 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.074480 (+ 4823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.076031 (+ 1551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.076042 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.076047 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.076239 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.076949 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.076956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.076960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.077577 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.078011 (+ 434us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.079499 (+ 1488us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.079876 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":517,"threads_started":1}
I20260806 20:25:39.080502 25037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:39.080638 24901 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.109.252:42373
I20260806 20:25:39.084089 25093 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 20:25:39.084585 24901 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.085080 24901 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:39.085311 24505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:39.092949 24952 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER af3861184a084cb482576e2fb6783735 (127.17.109.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.099107 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: NON_VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: true } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.099494 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: This master's current role is: LEARNER
I20260806 20:25:39.100903 25094 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: attempting to promote NON_VOTER c885244b0041460db0bc5fef74135f33 to VOTER
I20260806 20:25:39.101881 25094 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } }
I20260806 20:25:39.110586 25037 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Refusing update from remote peer f0eb02a82a7f41d4b87f3d54a1c243d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 20:25:39.110827 24952 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 LEARNER]: Refusing update from remote peer f0eb02a82a7f41d4b87f3d54a1c243d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 20:25:39.111299 24904 master.cc:537] Master@127.17.109.252:42373 shutting down...
I20260806 20:25:39.111620 24901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 20:25:39.112056 25035 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Refusing update from remote peer f0eb02a82a7f41d4b87f3d54a1c243d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 20:25:39.115286 25094 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c885244b0041460db0bc5fef74135f33 (127.17.109.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.116727 25098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:39.121193 25093 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.121577 25093 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
W20260806 20:25:39.122119 24294 consensus_peers.cc:487] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 -> Peer af3861184a084cb482576e2fb6783735 (127.17.109.252:42373): Couldn't send request to peer af3861184a084cb482576e2fb6783735. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 20:25:39.122200 24951 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c885244b0041460db0bc5fef74135f33 (127.17.109.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.128441 24980 sys_catalog.cc:455] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.128764 24980 sys_catalog.cc:458] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:25:39.120177 25094 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.129253 25094 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:39.130679 24904 raft_consensus.cc:2229] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:39.131126 24904 raft_consensus.cc:2258] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:39.131359 24904 tablet_replica.cc:330] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: stopping tablet replica
I20260806 20:25:39.138538 24904 master.cc:552] Master@127.17.109.252:42373 shutdown complete.
I20260806 20:25:39.138626 24904 master_runner.cc:305] Clearing existing system tablet
I20260806 20:25:39.139046 24904 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:39.139173 24904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:39.142606 24904 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 20:25:39.144605 25101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:39.144997 24904 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:39.145277 24904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 20:25:39.145393 24904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
uuid: "af3861184a084cb482576e2fb6783735"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "812c326f64662350c45215186c8b48f0"
server_key_iv: "2d3dcfcb4597ca1639efea49dfb82b1e"
server_key_version: "encryptionkey@0"
I20260806 20:25:39.151048 24904 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:39.158727 24904 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 20:25:39.158910 24904 log.cc:1196] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 20:25:39.159415 24904 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Deleting consensus metadata
I20260806 20:25:39.159837 24904 master_runner.cc:315] Copying system tablet from 127.17.109.254:40431
I20260806 20:25:39.162614 24904 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: tablet copy: Beginning tablet copy session from remote peer at address 127.17.109.254:40431
I20260806 20:25:39.183987 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.163530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.17.109.254:40431)
0806 20:25:39.163814 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.163822 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.168147 (+ 4325us) server_negotiation.cc:407] Connection header received
0806 20:25:39.168342 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.168349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.168453 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.168570 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.169801 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.171097 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.173138 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.173907 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.174034 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.176623 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.176631 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.176637 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.176691 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.180736 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.181590 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.181596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.181599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.181794 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.182265 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.182271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.182274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.182635 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.182809 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.183320 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.183553 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260806 20:25:39.184015 25107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.163455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34296)
0806 20:25:39.167998 (+ 4543us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.168027 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.168194 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.168621 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.168627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.169341 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:39.169613 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.169628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.171262 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.171269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.172970 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.172989 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.174097 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.174105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.174305 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.175137 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:39.175166 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:39.176383 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:39.180914 (+ 4531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.180920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.180925 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.181446 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.181932 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.181938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.181942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.182134 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.182795 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:39.182800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:39.183005 (+ 205us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.183164 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.183484 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4391,"thread_start_us":4288,"threads_started":1}
I20260806 20:25:39.185029 24331 tablet_copy_service.cc:140] P f0eb02a82a7f41d4b87f3d54a1c243d5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer af3861184a084cb482576e2fb6783735 ({username='oryx', principal='oryx/127.17.109.252@KRBTEST.COM'} at 127.0.0.1:34296)
I20260806 20:25:39.185206 24331 tablet_copy_service.cc:161] P f0eb02a82a7f41d4b87f3d54a1c243d5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer af3861184a084cb482576e2fb6783735 at {username='oryx', principal='oryx/127.17.109.252@KRBTEST.COM'} at 127.0.0.1:34296: session id = af3861184a084cb482576e2fb6783735-00000000000000000000000000000000
I20260806 20:25:39.189338 24331 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 20:25:39.191308 24904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 20:25:39.198431 24904 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: tablet copy: Starting download of 0 data blocks...
I20260806 20:25:39.198794 24904 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: tablet copy: Starting download of 1 WAL segments...
I20260806 20:25:39.201309 24904 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 20:25:39.205825 24331 tablet_copy_service.cc:342] P f0eb02a82a7f41d4b87f3d54a1c243d5: Request end of tablet copy session af3861184a084cb482576e2fb6783735-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.109.252@KRBTEST.COM'} at 127.0.0.1:34296
I20260806 20:25:39.205963 24331 tablet_copy_service.cc:433] P f0eb02a82a7f41d4b87f3d54a1c243d5: ending tablet copy session af3861184a084cb482576e2fb6783735-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer af3861184a084cb482576e2fb6783735
I20260806 20:25:39.217682 24904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:39.220522 25108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:39.221371 25109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:39.227154 24904 server_base.cc:1034] running on GCE node
I20260806 20:25:39.227198 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 24374
W20260806 20:25:39.231298 25111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:39.231783 24904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:39.232453 24904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:39.233654 24904 hybrid_clock.cc:648] HybridClock initialized: now 1786047939233611 us; error 36 us; skew 500 ppm
I20260806 20:25:39.233902 24904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:39 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047939, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:39.236276 24904 init.cc:377] Logged in from keytab as oryx/127.17.109.252@KRBTEST.COM (short username oryx)
I20260806 20:25:39.244029 24904 webserver.cc:460] Webserver started at http://127.17.109.252:45265/ using document root <none> and password file <none>
I20260806 20:25:39.244375 24904 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:39.244486 24904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:39.247740 24904 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 20:25:39.249734 25119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:39.250183 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:38705
--local_ip_for_outbound_sockets=127.17.109.193
--tserver_master_addrs=127.17.109.254:40431
--webserver_port=39973
--webserver_interface=127.17.109.193
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab
--principal=oryx/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779,127.17.109.252:42373 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:25:39.250265 24904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 20:25:39.250404 24904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
uuid: "af3861184a084cb482576e2fb6783735"
format_stamp: "Formatted at 2026-08-06 20:25:38 on dist-test-slave-6k22"
server_key: "812c326f64662350c45215186c8b48f0"
server_key_iv: "2d3dcfcb4597ca1639efea49dfb82b1e"
server_key_version: "encryptionkey@0"
I20260806 20:25:39.250573 24904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:39.284941 24904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:39.288543 24904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:39.314533 24904 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.252:42373
I20260806 20:25:39.314607 25173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.252:42373 every 8 connection(s)
I20260806 20:25:39.315479 24904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
I20260806 20:25:39.324151 25174 sys_catalog.cc:263] Verifying existing consensus state
I20260806 20:25:39.330549 25174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Bootstrap starting.
I20260806 20:25:39.346288 25174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:25:39.346786 25174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: Bootstrap complete.
I20260806 20:25:39.347720 25174 raft_consensus.cc:348] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } }
I20260806 20:25:39.347982 25174 raft_consensus.cc:729] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: af3861184a084cb482576e2fb6783735, State: Initialized, Role: LEARNER
I20260806 20:25:39.348250 25174 consensus_queue.cc:260] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } }
I20260806 20:25:39.350674 25177 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.351171 25177 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: This master's current role is: LEARNER
I20260806 20:25:39.351307 25174 sys_catalog.cc:564] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:39.380465 25188 catalog_manager.cc:1259] Loaded cluster ID: f7cb91e23f7e4e45b613e3176eff6d62
I20260806 20:25:39.380589 25188 catalog_manager.cc:1552] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: loading cluster ID for follower catalog manager: success
I20260806 20:25:39.384083 25188 catalog_manager.cc:1574] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: acquiring CA information for follower catalog manager: success
I20260806 20:25:39.386631 25188 catalog_manager.cc:1602] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 20:25:39.635491 25124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:39.635848 25124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:39.635953 25124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:39.644078 25124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:39.644230 25124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:39.644354 25124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:39.654573 25124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab
--principal=oryx/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:38705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=39973
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779,127.17.109.252:42373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:39.655927 25124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:39.657217 25124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:39.678380 25193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:39.678335 25196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:39.678457 25194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:39.680100 25124 server_base.cc:1034] running on GCE node
I20260806 20:25:39.680814 25124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:39.681766 25124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:39.683012 25124 hybrid_clock.cc:648] HybridClock initialized: now 1786047939682956 us; error 42 us; skew 500 ppm
I20260806 20:25:39.683337 25124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:39 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047939, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:39.687114 25124 init.cc:377] Logged in from keytab as oryx/127.17.109.193@KRBTEST.COM (short username oryx)
I20260806 20:25:39.688766 25124 webserver.cc:460] Webserver started at http://127.17.109.193:39973/ using document root <none> and password file <none>
I20260806 20:25:39.689468 25124 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:39.689605 25124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:39.695950 25124 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 20:25:39.698877 25204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:39.700373 25124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 20:25:39.700515 25124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "eda73047fa6741b3b9fe04c0c6eb62fa"
format_stamp: "Formatted at 2026-08-06 20:25:36 on dist-test-slave-6k22"
server_key: "0aea1f78740bb13cd4bd5ba9fc63d201"
server_key_iv: "23511ab326e56d40f8561c16f244ac00"
server_key_version: "encryptionkey@0"
I20260806 20:25:39.701546 25124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:39.718592 25124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:39.723335 25124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:39.723675 25124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:39.724659 25124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:39.725970 25124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:39.726058 25124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:39.726138 25124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:39.726210 25124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:39.738418 25213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.729620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:42373 (local address 127.0.0.1:57666)
0806 20:25:39.730197 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.730225 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.730413 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.731084 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.731091 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.731131 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:39.731385 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.731405 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.733560 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.733571 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.735462 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.735485 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.737390 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.737395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.737707 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.737763 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.737876 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.738037 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":244,"threads_started":1}
I20260806 20:25:39.738528 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.729872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.17.109.252:42373)
0806 20:25:39.730618 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.730627 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.730651 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:25:39.730739 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.730747 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.730868 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.731600 (+ 732us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:39.731875 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.733386 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.735697 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.737194 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.737413 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.737667 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.737948 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.738111 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":398,"threads_started":1}
I20260806 20:25:39.740272 25147 raft_consensus.cc:1203] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260806 20:25:39.740985 25177 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0eb02a82a7f41d4b87f3d54a1c243d5. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:39.741330 25177 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: This master's current role is: LEARNER
I20260806 20:25:39.765750 25124 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:38705
I20260806 20:25:39.765851 25324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:38705 every 8 connection(s)
I20260806 20:25:39.767202 25124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:39.773782 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 25124
I20260806 20:25:39.774190 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 24518
Aug 06 20:25:39 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.194@KRBTEST.COM
Aug 06 20:25:39 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047939, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.252@KRBTEST.COM
Aug 06 20:25:39 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047939, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
Aug 06 20:25:39 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047939, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.253@KRBTEST.COM
I20260806 20:25:39.835662 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.775826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.17.109.194:46033)
0806 20:25:39.808507 (+ 32681us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.808520 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.808560 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:25:39.808659 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.808672 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.809007 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.809203 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.815254 (+ 6051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.817418 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.819705 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.820564 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.820709 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.827781 (+ 7072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.827800 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.827822 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.827888 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.831897 (+ 4009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.832918 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.832928 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.832931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.833091 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.833547 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.833552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.833555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.834009 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.834323 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.834805 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.835089 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32441,"thread_start_us":33619,"threads_started":1}
I20260806 20:25:39.837711 24647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:39.839939 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.771701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:54007 (local address 127.17.109.254:40431)
0806 20:25:39.812016 (+ 40315us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.812026 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.812051 (+ 25us) server_negotiation.cc:407] Connection header received
0806 20:25:39.815866 (+ 3815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.815874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.815987 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.816105 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.819437 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.820638 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.824579 (+ 3941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.825250 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.825419 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.829842 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.829849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.829859 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.829917 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.833931 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.836370 (+ 2439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.836378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.836384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.836538 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.837094 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.837099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.837102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.837543 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.837682 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.839279 (+ 1597us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.839521 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40140,"thread_start_us":40000,"threads_started":1}
I20260806 20:25:39.840144 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.771475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:53567 (local address 127.17.109.252:42373)
0806 20:25:39.771747 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.771756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.812746 (+ 40990us) server_negotiation.cc:407] Connection header received
0806 20:25:39.816400 (+ 3654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.816406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.816489 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.816569 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.819019 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.820346 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.822913 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.823453 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.823670 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.829786 (+ 6116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.829795 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.829803 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.829854 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.834314 (+ 4460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.836396 (+ 2082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.836402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.836406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.836598 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.837206 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.837214 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.837218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.837766 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.838016 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.839135 (+ 1119us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.839484 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260806 20:25:39.841043 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.770184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:37553 (local address 127.17.109.253:46779)
0806 20:25:39.818245 (+ 48061us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:39.818254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:39.819176 (+ 922us) server_negotiation.cc:407] Connection header received
0806 20:25:39.819560 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:39.819583 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:39.819750 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:39.819853 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:39.821616 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.822759 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.826356 (+ 3597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.827320 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.827444 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.833209 (+ 5765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:39.833218 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:39.833225 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:39.833296 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:39.836886 (+ 3590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.837686 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.837692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.837697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.837856 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:39.838622 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:39.838627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:39.838630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:39.838980 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:39.839101 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:39.840475 (+ 1374us) server_negotiation.cc:299] Negotiation successful
0806 20:25:39.840698 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47936,"thread_start_us":48074,"threads_started":1}
I20260806 20:25:39.841688 25330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.770925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.193:54007)
0806 20:25:39.810564 (+ 39639us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.810631 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.815498 (+ 4867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.816394 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.816406 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.818929 (+ 2523us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:39.819201 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.819212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.820841 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.820848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.824345 (+ 3497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.824365 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.825760 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.825774 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.826070 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.826687 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:39.826713 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:39.829576 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:39.835470 (+ 5894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.835476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.835480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.836204 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.836689 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.836694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.836698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.836957 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.837782 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:39.837792 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:39.838321 (+ 529us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.838634 (+ 313us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.840441 (+ 1807us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39313,"mutex_wait_us":1494,"spinlock_wait_cycles":25472,"thread_start_us":39011,"threads_started":1}
I20260806 20:25:39.841779 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.770925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:42373 (local address 127.17.109.193:53567)
0806 20:25:39.812613 (+ 41688us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.812638 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.815669 (+ 3031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.816852 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.816857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.817576 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:39.818634 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.818671 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.820637 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.820649 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.822715 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.822746 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.823781 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.823795 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.824226 (+ 431us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.825075 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:39.825114 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:39.829441 (+ 4327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:39.834541 (+ 5100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.834554 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.834575 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.835536 (+ 961us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.836805 (+ 1269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.836810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.836814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.837024 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.838275 (+ 1251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:39.838288 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:39.838574 (+ 286us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.840344 (+ 1770us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.840737 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41508,"mutex_wait_us":948,"thread_start_us":46685,"threads_started":1}
I20260806 20:25:39.841719 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:39.772230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.17.109.193:37553)
0806 20:25:39.819048 (+ 46818us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:39.819072 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:39.819298 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:39.820114 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:39.820119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:39.820903 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:39.821418 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.821436 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.823928 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.823934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.826132 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:39.826151 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:39.827870 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:39.827879 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:39.828153 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:39.829183 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:39.829212 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:39.832927 (+ 3715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:39.837048 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.837052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.837056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.837543 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.837991 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:39.837996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:39.837999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:39.838482 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:39.839177 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:39.839184 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:39.839341 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 20:25:39.839467 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 20:25:39.839895 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46635,"mutex_wait_us":94,"spinlock_wait_cycles":26752,"thread_start_us":40103,"threads_started":1}
I20260806 20:25:39.843400 25326 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40431
I20260806 20:25:39.843432 25325 heartbeater.cc:347] Connected to a master server at 127.17.109.252:42373
I20260806 20:25:39.843506 25327 heartbeater.cc:347] Connected to a master server at 127.17.109.253:46779
I20260806 20:25:39.843890 25325 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:39.844669 25325 heartbeater.cc:511] Master 127.17.109.252:42373 requested a full tablet report, sending...
I20260806 20:25:39.843899 25326 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:39.843900 25327 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:39.845968 25326 heartbeater.cc:511] Master 127.17.109.254:40431 requested a full tablet report, sending...
I20260806 20:25:39.846160 25327 heartbeater.cc:511] Master 127.17.109.253:46779 requested a full tablet report, sending...
I20260806 20:25:39.847131 24311 ts_manager.cc:194] Re-registered known tserver with Master: eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705)
I20260806 20:25:39.847108 25137 ts_manager.cc:194] Registered new tserver with Master: eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705)
I20260806 20:25:39.848166 24311 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:54007
I20260806 20:25:39.848677 24942 ts_manager.cc:194] Registered new tserver with Master: eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705)
I20260806 20:25:39.935129 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 24518
I20260806 20:25:39.954820 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:46033
--local_ip_for_outbound_sockets=127.17.109.194
--tserver_master_addrs=127.17.109.254:40431
--webserver_port=44017
--webserver_interface=127.17.109.194
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab
--principal=oryx/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779,127.17.109.252:42373 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:40.298727 25337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:40.299052 25337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:40.299232 25337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:40.306689 25337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:40.306847 25337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:40.307026 25337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:40.316927 25337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab
--principal=oryx/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:46033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=44017
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779,127.17.109.252:42373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:40.318272 25337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:40.319687 25337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:40.333365 25343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:40.333318 25346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:40.333950 25337 server_base.cc:1034] running on GCE node
W20260806 20:25:40.333290 25344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:40.334725 25337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:40.335736 25337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:40.337047 25337 hybrid_clock.cc:648] HybridClock initialized: now 1786047940336973 us; error 44 us; skew 500 ppm
I20260806 20:25:40.337604 25337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:25:40.337888 25094 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: NON_VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: true } } }
I20260806 20:25:40.338306 25094 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:40.340664 25094 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: attempting to promote NON_VOTER af3861184a084cb482576e2fb6783735 to VOTER
I20260806 20:25:40.341439 25094 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } }
Aug 06 20:25:40 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047940, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:40.342537 25337 init.cc:377] Logged in from keytab as oryx/127.17.109.194@KRBTEST.COM (short username oryx)
I20260806 20:25:40.343879 25147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 LEARNER]: Refusing update from remote peer f0eb02a82a7f41d4b87f3d54a1c243d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 20:25:40.344352 24951 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Refusing update from remote peer f0eb02a82a7f41d4b87f3d54a1c243d5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260806 20:25:40.344981 25350 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 20:25:40.345414 25337 webserver.cc:460] Webserver started at http://127.17.109.194:44017/ using document root <none> and password file <none>
I20260806 20:25:40.345428 25094 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 20:25:40.346298 25337 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:40.346484 25337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:40.348315 25350 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, af3861184a084cb482576e2fb6783735 (127.17.109.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:40.348999 24951 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, af3861184a084cb482576e2fb6783735 (127.17.109.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:40.354916 25147 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, af3861184a084cb482576e2fb6783735 (127.17.109.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:40.356518 25337 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 20:25:40.359508 25094 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:40.359908 25094 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:40.359755 25349 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:40.360127 25349 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:40.360460 25350 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:40.360718 25350 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:40.361737 25363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:40.361737 25359 sys_catalog.cc:455] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:40.362200 25359 sys_catalog.cc:458] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:25:40.363811 25337 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260806 20:25:40.364037 25337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "a641597171fd4befae1ecec6faaf9dbc"
format_stamp: "Formatted at 2026-08-06 20:25:37 on dist-test-slave-6k22"
server_key: "0aee0f56da2414e862984d87c591308d"
server_key_iv: "ed3004dc669a23b113b141c00df58d62"
server_key_version: "encryptionkey@0"
I20260806 20:25:40.365176 25337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:40.367376 25362 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:40.367816 25362 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:25:40.403961 25337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:40.408398 25337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:40.408725 25337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:40.409762 25337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:40.411239 25337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:40.411327 25337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:40.411474 25337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:40.411588 25337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:40.446170 25337 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:46033
I20260806 20:25:40.446259 25483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:46033 every 8 connection(s)
I20260806 20:25:40.447803 25337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:40.452814 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 25337
I20260806 20:25:40.453147 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 24658
Aug 06 20:25:40 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047940, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
Aug 06 20:25:40 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047940, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.252@KRBTEST.COM
Aug 06 20:25:40 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047940, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.253@KRBTEST.COM
I20260806 20:25:40.490557 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:40.451541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:53211 (local address 127.17.109.254:40431)
0806 20:25:40.452356 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:40.452362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:40.453175 (+ 813us) server_negotiation.cc:407] Connection header received
0806 20:25:40.465179 (+ 12004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:40.465200 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:40.465338 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:40.465449 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:40.469411 (+ 3962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.470953 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.475756 (+ 4803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.476367 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.476513 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:40.482097 (+ 5584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:40.482102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:40.482106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:40.482140 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:40.486059 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:40.487222 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:40.487227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:40.487229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:40.487394 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:40.488452 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:40.488457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:40.488460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:40.488741 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:40.488909 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:40.489920 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0806 20:25:40.490145 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":767,"threads_started":1}
Aug 06 20:25:40 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.195@KRBTEST.COM
I20260806 20:25:40.491860 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:40.451820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.194:53211)
0806 20:25:40.453025 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:40.453071 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:40.462330 (+ 9259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:40.466402 (+ 4072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:40.466420 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:40.467533 (+ 1113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:40.468758 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:40.468792 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.471291 (+ 2499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.471299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:40.475606 (+ 4307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:40.475622 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.476637 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.476649 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:40.476981 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:40.477821 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:40.477852 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:40.481858 (+ 4006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:40.486663 (+ 4805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:40.486670 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:40.486684 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:40.487081 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:40.488093 (+ 1012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:40.488098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:40.488101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:40.488349 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:40.488881 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:40.488889 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:40.489354 (+ 465us) client_negotiation.cc:769] Sending connection context
0806 20:25:40.489748 (+ 394us) client_negotiation.cc:240] Negotiation successful
0806 20:25:40.490308 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":71,"spinlock_wait_cycles":6016,"thread_start_us":14983,"threads_started":1}
I20260806 20:25:40.492646 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:40.451162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:55055 (local address 127.17.109.252:42373)
0806 20:25:40.451864 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:40.451871 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:40.466270 (+ 14399us) server_negotiation.cc:407] Connection header received
0806 20:25:40.468170 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:40.468180 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:40.468296 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:40.468401 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:40.470472 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.471961 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.476767 (+ 4806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.477471 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.477579 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:40.483046 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:40.483053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:40.483058 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:40.483109 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:40.487234 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:40.488087 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:40.488093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:40.488097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:40.488263 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:40.489257 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:40.489261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:40.489264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:40.489612 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:40.489748 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:40.491775 (+ 2027us) server_negotiation.cc:299] Negotiation successful
0806 20:25:40.492301 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":551,"threads_started":1}
I20260806 20:25:40.493583 25485 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40431
I20260806 20:25:40.493870 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:40.452000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:42373 (local address 127.17.109.194:55055)
0806 20:25:40.466104 (+ 14104us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:40.466144 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:40.467925 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:40.468779 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:40.468787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:40.469735 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:40.470204 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:40.470220 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.472321 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.472330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:40.475581 (+ 3251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:40.475595 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.477926 (+ 2331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.477934 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:40.478140 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:40.478829 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:40.478849 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:40.481858 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:40.487372 (+ 5514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:40.487377 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:40.487380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:40.487953 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:40.488944 (+ 991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:40.488948 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:40.488951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:40.489103 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:40.491156 (+ 2053us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:40.491161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:40.491334 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 20:25:40.493101 (+ 1767us) client_negotiation.cc:240] Negotiation successful
0806 20:25:40.493323 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13906,"mutex_wait_us":996,"spinlock_wait_cycles":111232,"thread_start_us":509,"threads_started":1}
I20260806 20:25:40.493979 25485 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:40.494575 25492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:40.452175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.17.109.194:54607)
0806 20:25:40.474977 (+ 22802us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:40.474997 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:40.475122 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:40.475774 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:40.475786 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:40.476499 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:40.476966 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:40.476985 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.478930 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.478938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:40.480730 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:40.480745 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.482185 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.482192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:40.482387 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:40.483725 (+ 1338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:40.483749 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:40.486490 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:40.491039 (+ 4549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:40.491044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:40.491048 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:40.491672 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:40.492251 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:40.492257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:40.492260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:40.492466 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:40.493325 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:40.493331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:40.493555 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 20:25:40.493733 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 20:25:40.494060 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22662,"thread_start_us":22597,"threads_started":1}
I20260806 20:25:40.495185 25485 heartbeater.cc:511] Master 127.17.109.254:40431 requested a full tablet report, sending...
I20260806 20:25:40.495307 25484 heartbeater.cc:347] Connected to a master server at 127.17.109.252:42373
I20260806 20:25:40.495421 25484 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:40.495915 25484 heartbeater.cc:511] Master 127.17.109.252:42373 requested a full tablet report, sending...
I20260806 20:25:40.496927 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:40.452743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:54607 (local address 127.17.109.253:46779)
0806 20:25:40.474733 (+ 21990us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:40.474742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:40.475205 (+ 463us) server_negotiation.cc:407] Connection header received
0806 20:25:40.475290 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:40.475299 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:40.475394 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:40.475533 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:40.477269 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.478710 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.480925 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.481770 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.481905 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:40.486731 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:40.486738 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:40.486743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:40.486793 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:40.490856 (+ 4063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:40.491843 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:40.491851 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:40.491857 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:40.492079 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:40.492640 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:40.492647 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:40.492651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:40.493102 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:40.493259 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:40.496136 (+ 2877us) server_negotiation.cc:299] Negotiation successful
0806 20:25:40.496442 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21816,"thread_start_us":22275,"threads_started":1}
I20260806 20:25:40.496966 24311 ts_manager.cc:194] Re-registered known tserver with Master: a641597171fd4befae1ecec6faaf9dbc (127.17.109.194:46033)
I20260806 20:25:40.497116 25137 ts_manager.cc:194] Registered new tserver with Master: a641597171fd4befae1ecec6faaf9dbc (127.17.109.194:46033)
I20260806 20:25:40.498070 24311 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:53211
I20260806 20:25:40.498247 25486 heartbeater.cc:347] Connected to a master server at 127.17.109.253:46779
I20260806 20:25:40.498409 25486 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:40.498986 25486 heartbeater.cc:511] Master 127.17.109.253:46779 requested a full tablet report, sending...
I20260806 20:25:40.500283 24942 ts_manager.cc:194] Registered new tserver with Master: a641597171fd4befae1ecec6faaf9dbc (127.17.109.194:46033)
I20260806 20:25:40.502563 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:40.454678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53606 (local address 127.17.109.195:45435)
0806 20:25:40.467762 (+ 13084us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:40.467776 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:40.475058 (+ 7282us) server_negotiation.cc:407] Connection header received
0806 20:25:40.475395 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:40.475407 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:40.475731 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:40.475917 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:40.479016 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.480750 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.482793 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:40.483758 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:40.483904 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:40.494011 (+ 10107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:40.494033 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:40.494049 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:40.494108 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:40.498663 (+ 4555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:40.499542 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:40.499552 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:40.499578 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:40.500652 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:40.501090 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:40.501094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:40.501096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:40.501457 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:40.501707 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:40.501975 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 20:25:40.502199 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12822,"thread_start_us":14311,"threads_started":1}
I20260806 20:25:40.503831 24787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:40.589397 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 24658
I20260806 20:25:40.610013 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:45435
--local_ip_for_outbound_sockets=127.17.109.195
--tserver_master_addrs=127.17.109.254:40431
--webserver_port=45971
--webserver_interface=127.17.109.195
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab
--principal=oryx/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779,127.17.109.252:42373 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:25:40.850545 25326 heartbeater.cc:503] Master 127.17.109.254:40431 was elected leader, sending a full tablet report...
W20260806 20:25:40.954275 25497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:40.954592 25497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:40.954752 25497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:40.962146 25497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:40.962314 25497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:40.962512 25497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:40.972653 25497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab
--principal=oryx/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:45435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=45971
--tserver_master_addrs=127.17.109.254:40431,127.17.109.253:46779,127.17.109.252:42373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:40.974097 25497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:40.975495 25497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:40.989851 25505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:40.989912 25502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:40.989943 25503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:40.990756 25497 server_base.cc:1034] running on GCE node
I20260806 20:25:40.991479 25497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:40.992558 25497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:40.993801 25497 hybrid_clock.cc:648] HybridClock initialized: now 1786047940993737 us; error 57 us; skew 500 ppm
I20260806 20:25:40.994140 25497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:40 dist-test-slave-6k22 krb5kdc[24265](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047940, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:41.003953 25497 init.cc:377] Logged in from keytab as oryx/127.17.109.195@KRBTEST.COM (short username oryx)
I20260806 20:25:41.005872 25497 webserver.cc:460] Webserver started at http://127.17.109.195:45971/ using document root <none> and password file <none>
I20260806 20:25:41.006677 25497 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:41.006855 25497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:41.012965 25497 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 20:25:41.015971 25512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:41.017383 25497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:41.017577 25497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "cf387168244743de91a29001aad33725"
format_stamp: "Formatted at 2026-08-06 20:25:37 on dist-test-slave-6k22"
server_key: "bf73ad20b473d2313203b12914f46e45"
server_key_iv: "f493dec9471c444f84968d5e7aac7158"
server_key_version: "encryptionkey@0"
I20260806 20:25:41.018690 25497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:41.071108 25497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:41.076299 25497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:41.076646 25497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:41.077785 25497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:41.079447 25497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:41.079604 25497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:41.079763 25497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:41.079866 25497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:41.117238 25497 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:45435
I20260806 20:25:41.117300 25633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:45435 every 8 connection(s)
I20260806 20:25:41.118795 25497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:41.127281 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 25497
Aug 06 20:25:41 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047940, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.252@KRBTEST.COM
Aug 06 20:25:41 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047940, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.253@KRBTEST.COM
Aug 06 20:25:41 dist-test-slave-6k22 krb5kdc[24265](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047940, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:41.180135 25639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.122172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:51501 (local address 127.17.109.254:40431)
0806 20:25:41.146504 (+ 24332us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.146512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.146911 (+ 399us) server_negotiation.cc:407] Connection header received
0806 20:25:41.146989 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.146997 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.147083 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.147309 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.152556 (+ 5247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.154168 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.157194 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.157875 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.158006 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.169989 (+ 11983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.169995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.170000 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.170058 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.174490 (+ 4432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.175898 (+ 1408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.175904 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.175911 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.176101 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.177153 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.177158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.177162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.177580 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.177734 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.179442 (+ 1708us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.179709 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24206,"thread_start_us":24098,"threads_started":1}
I20260806 20:25:41.181538 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.122595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.195:51501)
0806 20:25:41.144556 (+ 21961us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:41.144572 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:41.144656 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:41.147358 (+ 2702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:41.147363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:41.148178 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:41.151223 (+ 3045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.151353 (+ 130us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.154347 (+ 2994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.154355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.157007 (+ 2652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.157026 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.158109 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.158118 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.158524 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.164809 (+ 6285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:41.164841 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:41.169640 (+ 4799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:41.174963 (+ 5323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:41.174977 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:41.174999 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:41.175763 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:41.176714 (+ 951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:41.176720 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:41.176724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:41.177014 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:41.178310 (+ 1296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:41.178319 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:41.178900 (+ 581us) client_negotiation.cc:769] Sending connection context
0806 20:25:41.179287 (+ 387us) client_negotiation.cc:240] Negotiation successful
0806 20:25:41.179934 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21819,"mutex_wait_us":290,"thread_start_us":14275,"threads_started":1}
I20260806 20:25:41.183588 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.122756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.17.109.195:54941)
0806 20:25:41.143789 (+ 21033us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:41.143817 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:41.143974 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:41.146382 (+ 2408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:41.146396 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:41.148067 (+ 1671us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:41.149308 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.149322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.155808 (+ 6486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.155814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.157347 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.157363 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.158463 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.158471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.158738 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.162520 (+ 3782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:41.162570 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:41.167153 (+ 4583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:41.175967 (+ 8814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:41.175974 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:41.175978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:41.176577 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:41.181432 (+ 4855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:41.181439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:41.181444 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:41.181643 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:41.182388 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:41.182392 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:41.182570 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 20:25:41.182731 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 20:25:41.183088 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20859,"mutex_wait_us":168,"thread_start_us":21510,"threads_started":1}
I20260806 20:25:41.184180 25635 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40431
I20260806 20:25:41.184559 25635 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:41.184690 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.121832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:54941 (local address 127.17.109.253:46779)
0806 20:25:41.145428 (+ 23596us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.145434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.145458 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:25:41.145529 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.145535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.145646 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.146009 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.149516 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.154887 (+ 5371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.157562 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.158251 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.158421 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.167386 (+ 8965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.167403 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.167412 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.167463 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.174797 (+ 7334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.181113 (+ 6316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.181119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.181123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.181291 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.181813 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.181821 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.181826 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.182229 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.183866 (+ 1637us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.184029 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.184283 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23444,"thread_start_us":23307,"threads_started":1}
I20260806 20:25:41.185570 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.122157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:45675 (local address 127.17.109.252:42373)
0806 20:25:41.146200 (+ 24043us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.146209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.146238 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:25:41.146317 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.146321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.146427 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.146583 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.150751 (+ 4168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.155513 (+ 4762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.161223 (+ 5710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.162015 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.162172 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.167160 (+ 4988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.167166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.167171 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.167231 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.177018 (+ 9787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.180815 (+ 3797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.180821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.180825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.181008 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.181541 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.181546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.181550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.182588 (+ 1038us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.184269 (+ 1681us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.184529 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.185139 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23877,"thread_start_us":24566,"threads_started":1}
I20260806 20:25:41.185604 25635 heartbeater.cc:511] Master 127.17.109.254:40431 requested a full tablet report, sending...
I20260806 20:25:41.185840 25636 heartbeater.cc:347] Connected to a master server at 127.17.109.253:46779
I20260806 20:25:41.186160 25636 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:41.186774 25636 heartbeater.cc:511] Master 127.17.109.253:46779 requested a full tablet report, sending...
I20260806 20:25:41.187840 24311 ts_manager.cc:194] Re-registered known tserver with Master: cf387168244743de91a29001aad33725 (127.17.109.195:45435)
I20260806 20:25:41.188704 24942 ts_manager.cc:194] Registered new tserver with Master: cf387168244743de91a29001aad33725 (127.17.109.195:45435)
I20260806 20:25:41.189286 24311 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:51501
I20260806 20:25:41.190538 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.122595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:42373 (local address 127.17.109.195:45675)
0806 20:25:41.138957 (+ 16362us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:41.139044 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:41.142782 (+ 3738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:41.146610 (+ 3828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:41.146616 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:41.147712 (+ 1096us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:41.148956 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.148991 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.158671 (+ 9680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.158677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.160971 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.161007 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.162222 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.162229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.162454 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.163178 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:41.163221 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:41.166758 (+ 3537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:41.178327 (+ 11569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:41.178331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:41.178335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:41.179183 (+ 848us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:41.181160 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:41.181166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:41.181170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:41.181364 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:41.182767 (+ 1403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:41.182775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:41.183004 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 20:25:41.183155 (+ 151us) client_negotiation.cc:240] Negotiation successful
0806 20:25:41.183478 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15933,"mutex_wait_us":74,"thread_start_us":20679,"threads_started":1}
I20260806 20:25:41.191529 25634 heartbeater.cc:347] Connected to a master server at 127.17.109.252:42373
I20260806 20:25:41.191713 25634 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:41.192364 25634 heartbeater.cc:511] Master 127.17.109.252:42373 requested a full tablet report, sending...
I20260806 20:25:41.193672 25137 ts_manager.cc:194] Registered new tserver with Master: cf387168244743de91a29001aad33725 (127.17.109.195:45435)
I20260806 20:25:41.194909 25649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.148924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.17.109.252:42373)
0806 20:25:41.165865 (+ 16941us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.165870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.165896 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:25:41.165964 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.165969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.166055 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.166221 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.172807 (+ 6586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.174104 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.176174 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.176947 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.177041 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.184168 (+ 7127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.184177 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.184184 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.184223 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.190454 (+ 6231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.191372 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.191377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.191380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.191546 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.192672 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.192677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.192680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.193117 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.193334 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.194270 (+ 936us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.194511 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16800,"thread_start_us":17994,"threads_started":1}
I20260806 20:25:41.195777 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.159854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34298 (local address 127.17.109.254:40431)
0806 20:25:41.160417 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.160424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.167980 (+ 7556us) server_negotiation.cc:407] Connection header received
0806 20:25:41.168053 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.168061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.168158 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.168399 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.171204 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.172801 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.177174 (+ 4373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.177949 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.178086 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.186675 (+ 8589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.186682 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.186687 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.186729 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.191603 (+ 4874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.192636 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.192645 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.192649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.192841 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.193977 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.193985 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.193989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.194367 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.194622 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.194929 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.195169 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":290,"threads_started":1}
I20260806 20:25:41.197700 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.148930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59072 (local address 127.17.109.253:46779)
0806 20:25:41.166780 (+ 17850us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.166786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.166805 (+ 19us) server_negotiation.cc:407] Connection header received
0806 20:25:41.166856 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.166863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.166957 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.169376 (+ 2419us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.171288 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.172764 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.176010 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.176689 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.177888 (+ 1199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.182849 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.182855 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.182860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.182894 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.193537 (+ 10643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.195379 (+ 1842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.195383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.195385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.195610 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.196083 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.196088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.196090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.196419 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.196657 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.197137 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.197378 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17724,"thread_start_us":18668,"threads_started":1}
I20260806 20:25:41.200017 24311 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:25:41.207630 24311 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:41.236860 25662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.227050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:38705 (local address 127.0.0.1:38316)
0806 20:25:41.227825 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:41.227850 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:41.227976 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:41.228784 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:41.228789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:41.228822 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:41.229299 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.229312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.230774 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.230783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.233313 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.233334 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.236100 (+ 2766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.236105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.236231 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.236254 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 20:25:41.236363 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 20:25:41.236489 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":415,"threads_started":1}
I20260806 20:25:41.236894 25663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.227440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.17.109.193:38705)
0806 20:25:41.228069 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.228082 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.228113 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:25:41.228228 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.228237 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.228526 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.229346 (+ 820us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:41.229568 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.230628 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.233539 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.235520 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.235797 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.235979 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.236410 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.236578 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":302,"threads_started":1}
I20260806 20:25:41.238358 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.228240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:46033 (local address 127.0.0.1:39682)
0806 20:25:41.228819 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:41.228846 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:41.228955 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:41.232087 (+ 3132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:41.232094 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:41.232122 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:41.232462 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.232472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.234043 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.234051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.235346 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.235357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.237509 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.237515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.237696 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.237726 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 20:25:41.237822 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 20:25:41.237947 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":279,"threads_started":1}
I20260806 20:25:41.239090 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.228527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39682 (local address 127.17.109.194:46033)
0806 20:25:41.231476 (+ 2949us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.231487 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.231517 (+ 30us) server_negotiation.cc:407] Connection header received
0806 20:25:41.231634 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.231642 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.231889 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.232093 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:41.232652 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.233901 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.235490 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.237239 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.237486 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.237660 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.238643 (+ 983us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.238820 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2787,"thread_start_us":2688,"threads_started":1}
I20260806 20:25:41.240722 25256 tablet_service.cc:1467] Processing CreateTablet for tablet a4d782b8e69d4e59a200209547282e6e (DEFAULT_TABLE table=test-table [id=38ae39614d2e461c9496614d56a62607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:41.242552 25256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4d782b8e69d4e59a200209547282e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:41.242420 25417 tablet_service.cc:1467] Processing CreateTablet for tablet a4d782b8e69d4e59a200209547282e6e (DEFAULT_TABLE table=test-table [id=38ae39614d2e461c9496614d56a62607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:41.243737 25417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4d782b8e69d4e59a200209547282e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:41.246465 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.231332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:45435 (local address 127.0.0.1:53620)
0806 20:25:41.231922 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:41.231943 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:41.232058 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:41.237989 (+ 5931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:41.237993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:41.238019 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:41.238453 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.238468 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.241412 (+ 2944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.241417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.243491 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.243506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.245685 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.245692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.245874 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.245911 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 20:25:41.246008 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 20:25:41.246139 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":359,"threads_started":1}
I20260806 20:25:41.246592 25667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.232437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53620 (local address 127.17.109.195:45435)
0806 20:25:41.236091 (+ 3654us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.236099 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.236130 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:25:41.236226 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.236234 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.237780 (+ 1546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.238009 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:41.238789 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.241191 (+ 2402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.243941 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.245487 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.245657 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.245821 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.246066 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.246238 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3485,"thread_start_us":3324,"threads_started":1}
I20260806 20:25:41.250088 25567 tablet_service.cc:1467] Processing CreateTablet for tablet a4d782b8e69d4e59a200209547282e6e (DEFAULT_TABLE table=test-table [id=38ae39614d2e461c9496614d56a62607]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:41.251230 25567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4d782b8e69d4e59a200209547282e6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:41.257043 25668 tablet_bootstrap.cc:492] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Bootstrap starting.
I20260806 20:25:41.260264 25668 tablet_bootstrap.cc:654] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:41.261312 25668 log.cc:824] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:41.264910 25672 tablet_bootstrap.cc:492] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Bootstrap starting.
I20260806 20:25:41.265594 25668 tablet_bootstrap.cc:492] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: No bootstrap required, opened a new log
I20260806 20:25:41.265874 25668 ts_tablet_manager.cc:1397] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260806 20:25:41.266590 25671 tablet_bootstrap.cc:492] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Bootstrap starting.
I20260806 20:25:41.268977 25672 tablet_bootstrap.cc:654] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:41.269775 25671 tablet_bootstrap.cc:654] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:41.270366 25672 log.cc:824] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:41.270926 25671 log.cc:824] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:41.273725 25672 tablet_bootstrap.cc:492] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: No bootstrap required, opened a new log
I20260806 20:25:41.273881 25671 tablet_bootstrap.cc:492] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: No bootstrap required, opened a new log
I20260806 20:25:41.274016 25672 ts_tablet_manager.cc:1397] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260806 20:25:41.274108 25671 ts_tablet_manager.cc:1397] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260806 20:25:41.275895 25668 raft_consensus.cc:348] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.276245 25668 raft_consensus.cc:374] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:41.276353 25668 raft_consensus.cc:729] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a641597171fd4befae1ecec6faaf9dbc, State: Initialized, Role: FOLLOWER
I20260806 20:25:41.276937 25668 consensus_queue.cc:260] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.280860 25485 heartbeater.cc:503] Master 127.17.109.254:40431 was elected leader, sending a full tablet report...
I20260806 20:25:41.282773 25668 ts_tablet_manager.cc:1428] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Time spent starting tablet: real 0.017s user 0.003s sys 0.009s
I20260806 20:25:41.284946 25671 raft_consensus.cc:348] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.285005 25672 raft_consensus.cc:348] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.285413 25671 raft_consensus.cc:374] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:41.285413 25672 raft_consensus.cc:374] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:41.285519 25672 raft_consensus.cc:729] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf387168244743de91a29001aad33725, State: Initialized, Role: FOLLOWER
I20260806 20:25:41.285519 25671 raft_consensus.cc:729] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda73047fa6741b3b9fe04c0c6eb62fa, State: Initialized, Role: FOLLOWER
I20260806 20:25:41.286170 25671 consensus_queue.cc:260] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.286237 25672 consensus_queue.cc:260] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.288472 25671 ts_tablet_manager.cc:1428] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260806 20:25:41.288735 25672 ts_tablet_manager.cc:1428] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260806 20:25:41.289019 25635 heartbeater.cc:503] Master 127.17.109.254:40431 was elected leader, sending a full tablet report...
W20260806 20:25:41.314822 25328 tablet.cc:2369] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:25:41.371862 25637 tablet.cc:2369] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:25:41.450982 25487 tablet.cc:2369] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:25:41.621085 25675 raft_consensus.cc:482] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:41.621436 25675 raft_consensus.cc:504] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.623445 25675 leader_election.cc:288] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705), cf387168244743de91a29001aad33725 (127.17.109.195:45435)
I20260806 20:25:41.630437 25667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.624073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:47505 (local address 127.17.109.195:45435)
0806 20:25:41.624482 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.624486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.625164 (+ 678us) server_negotiation.cc:407] Connection header received
0806 20:25:41.625396 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.625402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.625467 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.625559 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:41.626121 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.626828 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.628725 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.629517 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.629694 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.629780 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.630047 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.630176 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260806 20:25:41.630550 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.623835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:45435 (local address 127.17.109.194:47505)
0806 20:25:41.625043 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:41.625060 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:41.625245 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:41.625577 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:41.625582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:41.625643 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:41.625982 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.625990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.626962 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.626968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.628544 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.628556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.629751 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.629757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.629895 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.629918 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 20:25:41.630021 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 20:25:41.630147 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"spinlock_wait_cycles":10624,"thread_start_us":838,"threads_started":1}
I20260806 20:25:41.631453 25587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4d782b8e69d4e59a200209547282e6e" candidate_uuid: "a641597171fd4befae1ecec6faaf9dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf387168244743de91a29001aad33725" is_pre_election: true
I20260806 20:25:41.631908 25663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.624035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:51175 (local address 127.17.109.193:38705)
0806 20:25:41.624366 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.624375 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.625567 (+ 1192us) server_negotiation.cc:407] Connection header received
0806 20:25:41.625821 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.625826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.625902 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.626014 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:41.626668 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.627473 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.629765 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.630811 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.631019 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.631114 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.631430 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.631606 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260806 20:25:41.632050 25679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.624150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:38705 (local address 127.17.109.194:51175)
0806 20:25:41.625425 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:41.625442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:41.625644 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:41.626057 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:41.626063 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:41.626094 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:41.626468 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.626479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.627723 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.627730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.629576 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:41.629591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.631057 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.631064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:41.631252 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.631280 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 20:25:41.631401 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 20:25:41.631541 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"spinlock_wait_cycles":896,"thread_start_us":870,"threads_started":1}
I20260806 20:25:41.632054 25587 raft_consensus.cc:2454] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a641597171fd4befae1ecec6faaf9dbc in term 0.
I20260806 20:25:41.632715 25367 leader_election.cc:302] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a641597171fd4befae1ecec6faaf9dbc, cf387168244743de91a29001aad33725; no voters:
I20260806 20:25:41.633080 25675 raft_consensus.cc:2781] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:41.633227 25675 raft_consensus.cc:482] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:41.633344 25675 raft_consensus.cc:3037] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:41.633178 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4d782b8e69d4e59a200209547282e6e" candidate_uuid: "a641597171fd4befae1ecec6faaf9dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" is_pre_election: true
I20260806 20:25:41.633831 25276 raft_consensus.cc:2454] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a641597171fd4befae1ecec6faaf9dbc in term 0.
I20260806 20:25:41.635816 25675 raft_consensus.cc:504] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.636399 25675 leader_election.cc:288] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [CANDIDATE]: Term 1 election: Requested vote from peers eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705), cf387168244743de91a29001aad33725 (127.17.109.195:45435)
I20260806 20:25:41.636893 25587 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4d782b8e69d4e59a200209547282e6e" candidate_uuid: "a641597171fd4befae1ecec6faaf9dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf387168244743de91a29001aad33725"
I20260806 20:25:41.636904 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4d782b8e69d4e59a200209547282e6e" candidate_uuid: "a641597171fd4befae1ecec6faaf9dbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa"
I20260806 20:25:41.637113 25276 raft_consensus.cc:3037] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:41.637117 25587 raft_consensus.cc:3037] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:41.639653 25587 raft_consensus.cc:2454] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a641597171fd4befae1ecec6faaf9dbc in term 1.
I20260806 20:25:41.640033 25367 leader_election.cc:302] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a641597171fd4befae1ecec6faaf9dbc, cf387168244743de91a29001aad33725; no voters:
I20260806 20:25:41.640061 25276 raft_consensus.cc:2454] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a641597171fd4befae1ecec6faaf9dbc in term 1.
I20260806 20:25:41.640319 25675 raft_consensus.cc:2781] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:41.640949 25675 raft_consensus.cc:686] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 1 LEADER]: Becoming Leader. State: Replica: a641597171fd4befae1ecec6faaf9dbc, State: Running, Role: LEADER
I20260806 20:25:41.641737 25675 consensus_queue.cc:237] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:41.648036 24311 catalog_manager.cc:5495] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc reported cstate change: term changed from 0 to 1, leader changed from <none> to a641597171fd4befae1ecec6faaf9dbc (127.17.109.194). New cstate: current_term: 1 leader_uuid: "a641597171fd4befae1ecec6faaf9dbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } health_report { overall_health: UNKNOWN } } }
I20260806 20:25:41.691612 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.685868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.17.109.194:46033)
0806 20:25:41.686096 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.686111 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.686138 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:25:41.686285 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.686291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.686430 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.686614 (+ 184us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:41.687099 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.688111 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.689489 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.689891 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.690002 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.690440 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:41.690959 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:41.691102 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.691141 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.691230 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 20:25:41.700860 25587 raft_consensus.cc:1261] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 1 FOLLOWER]: Refusing update from remote peer a641597171fd4befae1ecec6faaf9dbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:41.700858 25276 raft_consensus.cc:1261] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 1 FOLLOWER]: Refusing update from remote peer a641597171fd4befae1ecec6faaf9dbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:41.701834 25680 consensus_queue.cc:1035] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:41.702216 25675 consensus_queue.cc:1035] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:41.720065 25684 mvcc.cc:205] Tried to move back new op lower bound from 7315652369196056576 to 7315652368968589312. Current Snapshot: MvccSnapshot[applied={T|T < 7315652369196056576}]
I20260806 20:25:41.731968 25685 mvcc.cc:205] Tried to move back new op lower bound from 7315652369196056576 to 7315652368968589312. Current Snapshot: MvccSnapshot[applied={T|T < 7315652369196056576 or (T in {7315652369196056576})}]
I20260806 20:25:41.736338 24310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:41.736624 24310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:41.744294 24310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0eb02a82a7f41d4b87f3d54a1c243d5: Sending DeleteTablet for 3 replicas of tablet a4d782b8e69d4e59a200209547282e6e
I20260806 20:25:41.745474 25256 tablet_service.cc:1514] Processing DeleteTablet for tablet a4d782b8e69d4e59a200209547282e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:41 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:38316
I20260806 20:25:41.745783 25417 tablet_service.cc:1514] Processing DeleteTablet for tablet a4d782b8e69d4e59a200209547282e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:41 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:39682
I20260806 20:25:41.746102 25567 tablet_service.cc:1514] Processing DeleteTablet for tablet a4d782b8e69d4e59a200209547282e6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:41 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:53620
I20260806 20:25:41.776252 25693 tablet_replica.cc:330] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: stopping tablet replica
I20260806 20:25:41.776925 25693 raft_consensus.cc:2229] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:41.777824 25693 raft_consensus.cc:2258] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:41.780702 25695 tablet_replica.cc:330] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: stopping tablet replica
I20260806 20:25:41.781848 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 24278
I20260806 20:25:41.781278 25695 raft_consensus.cc:2229] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:41.783072 25695 raft_consensus.cc:2258] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:41.783258 24342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:41.785408 25695 ts_tablet_manager.cc:1905] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:41.786013 25693 ts_tablet_manager.cc:1905] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:41.790524 25692 tablet_replica.cc:330] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: stopping tablet replica
I20260806 20:25:41.791076 25692 raft_consensus.cc:2229] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:41.795413 25692 raft_consensus.cc:2258] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:41.798938 25692 ts_tablet_manager.cc:1905] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:41.803110 25695 ts_tablet_manager.cc:1918] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:41.804625 25693 ts_tablet_manager.cc:1918] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:41.804961 25693 log.cc:1196] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/a4d782b8e69d4e59a200209547282e6e
I20260806 20:25:41.805404 25695 log.cc:1196] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/a4d782b8e69d4e59a200209547282e6e
I20260806 20:25:41.806627 25693 ts_tablet_manager.cc:1939] T a4d782b8e69d4e59a200209547282e6e P a641597171fd4befae1ecec6faaf9dbc: Deleting consensus metadata
I20260806 20:25:41.806983 25695 ts_tablet_manager.cc:1939] T a4d782b8e69d4e59a200209547282e6e P cf387168244743de91a29001aad33725: Deleting consensus metadata
I20260806 20:25:41.810935 25692 ts_tablet_manager.cc:1918] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:41.811141 25692 log.cc:1196] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/a4d782b8e69d4e59a200209547282e6e
I20260806 20:25:41.811892 25692 ts_tablet_manager.cc:1939] T a4d782b8e69d4e59a200209547282e6e P eda73047fa6741b3b9fe04c0c6eb62fa: Deleting consensus metadata
I20260806 20:25:41.821362 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.751693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59080 (local address 127.17.109.253:46779)
0806 20:25:41.779504 (+ 27811us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.779514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.779546 (+ 32us) server_negotiation.cc:407] Connection header received
0806 20:25:41.779655 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.779664 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.779797 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.779958 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.791681 (+ 11723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.793021 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.796259 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.797054 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.797188 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.812950 (+ 15762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.812960 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.812968 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.813021 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.817358 (+ 4337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.818532 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.818539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.818543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.818849 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.819372 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.819378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.819382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.819861 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.820115 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.820705 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.820984 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27613,"thread_start_us":28459,"threads_started":1}
I20260806 20:25:41.838793 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.776528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57688 (local address 127.17.109.252:42373)
0806 20:25:41.789776 (+ 13248us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.789786 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.822449 (+ 32663us) server_negotiation.cc:407] Connection header received
0806 20:25:41.822602 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.822607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.822716 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.822852 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.824729 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.825917 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.827850 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.828574 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.828701 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.832284 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.832291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.832296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.832339 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.835860 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.836673 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.836679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.836683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.836841 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.837252 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.837257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.837267 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.837587 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.837885 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.838175 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.838416 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12944,"thread_start_us":12826,"threads_started":1}
I20260806 20:25:41.933214 24295 catalog_manager.cc:4841] TS eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705): tablet a4d782b8e69d4e59a200209547282e6e (table test-table [id=38ae39614d2e461c9496614d56a62607]) successfully deleted
I20260806 20:25:41.933702 24298 catalog_manager.cc:4841] TS cf387168244743de91a29001aad33725 (127.17.109.195:45435): tablet a4d782b8e69d4e59a200209547282e6e (table test-table [id=38ae39614d2e461c9496614d56a62607]) successfully deleted
I20260806 20:25:41.933957 24298 catalog_manager.cc:4841] TS a641597171fd4befae1ecec6faaf9dbc (127.17.109.194:46033): tablet a4d782b8e69d4e59a200209547282e6e (table test-table [id=38ae39614d2e461c9496614d56a62607]) successfully deleted
I20260806 20:25:41.938900 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 24278
I20260806 20:25:41.997541 25697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.971291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59086 (local address 127.17.109.253:46779)
0806 20:25:41.971496 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.971503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.979414 (+ 7911us) server_negotiation.cc:407] Connection header received
0806 20:25:41.981339 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.981348 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.981470 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.981601 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.985340 (+ 3739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.986680 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.988583 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.989270 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.989402 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.992032 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.992038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.992042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.992089 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.994914 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.995588 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.995597 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.995602 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.995733 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.996120 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.996125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.996129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.996425 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.996690 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.997059 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.997302 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 20:25:41.999595 25698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:41.971871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.17.109.252:42373)
0806 20:25:41.972168 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:41.972178 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:41.979277 (+ 7099us) server_negotiation.cc:407] Connection header received
0806 20:25:41.979953 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:41.979964 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:41.980100 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:41.980258 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:41.982942 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.984226 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.986907 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:41.987689 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:41.987836 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:41.992500 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:41.992507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:41.992512 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:41.992558 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:41.996095 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.997021 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.997027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.997030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.997204 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:41.997871 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:41.997877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:41.997881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:41.998271 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:41.998534 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:41.999143 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 20:25:41.999320 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 20:25:42.729384 25711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:42.728420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.194:55513)
0806 20:25:42.728926 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:42.729087 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":276,"threads_started":1}
W20260806 20:25:42.729964 25485 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:40431 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
I20260806 20:25:42.733199 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:42.732361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.193:44287)
0806 20:25:42.732870 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:42.733012 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":234,"threads_started":1}
I20260806 20:25:42.733237 25713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:42.732597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.195:52223)
0806 20:25:42.732987 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:42.733095 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":209,"threads_started":1}
W20260806 20:25:42.733618 25635 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:40431 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
W20260806 20:25:42.733947 25326 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:40431 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
I20260806 20:25:43.244838 25714 raft_consensus.cc:482] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f0eb02a82a7f41d4b87f3d54a1c243d5)
I20260806 20:25:43.245101 25714 raft_consensus.cc:504] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } }
I20260806 20:25:43.246089 25715 raft_consensus.cc:482] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f0eb02a82a7f41d4b87f3d54a1c243d5)
I20260806 20:25:43.246345 25715 raft_consensus.cc:504] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } }
I20260806 20:25:43.246585 25714 leader_election.cc:288] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f0eb02a82a7f41d4b87f3d54a1c243d5 (127.17.109.254:40431), af3861184a084cb482576e2fb6783735 (127.17.109.252:42373)
I20260806 20:25:43.262786 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.246680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34716)
0806 20:25:43.262128 (+ 15448us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.262427 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15259,"threads_started":1}
I20260806 20:25:43.263082 25715 leader_election.cc:288] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f0eb02a82a7f41d4b87f3d54a1c243d5 (127.17.109.254:40431), c885244b0041460db0bc5fef74135f33 (127.17.109.253:46779)
W20260806 20:25:43.264935 24927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
I20260806 20:25:43.265175 25721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.263523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34728)
0806 20:25:43.264676 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.264890 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1012,"thread_start_us":832,"threads_started":1}
W20260806 20:25:43.265537 25122 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
I20260806 20:25:43.270292 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.269835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34738)
0806 20:25:43.270014 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.270117 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 20:25:43.270525 24927 leader_election.cc:334] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f0eb02a82a7f41d4b87f3d54a1c243d5 (127.17.109.254:40431): Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
I20260806 20:25:43.272060 25721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.271423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34752)
0806 20:25:43.271760 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.271875 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 20:25:43.272302 25122 leader_election.cc:334] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f0eb02a82a7f41d4b87f3d54a1c243d5 (127.17.109.254:40431): Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
I20260806 20:25:43.272944 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.247004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:42373 (local address 127.0.0.1:57706)
0806 20:25:43.262485 (+ 15481us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.262514 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:43.262734 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:43.264312 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:43.264320 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:43.264367 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:43.264757 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.264774 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.267280 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.267289 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.270307 (+ 3018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.270319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.272001 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.272008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.272259 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.272314 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 20:25:43.272446 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 20:25:43.272606 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15267,"thread_start_us":14917,"threads_started":1}
I20260806 20:25:43.273183 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.247900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.17.109.252:42373)
0806 20:25:43.263804 (+ 15904us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.263814 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.263843 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:25:43.263939 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.263947 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.264101 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.264939 (+ 838us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:43.265330 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.267046 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.270509 (+ 3463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.271800 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.271958 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.272052 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.272530 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.272668 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":15714,"thread_start_us":15256,"threads_started":1}
I20260806 20:25:43.273965 25147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c885244b0041460db0bc5fef74135f33" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "af3861184a084cb482576e2fb6783735" is_pre_election: true
I20260806 20:25:43.274333 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.248408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:46779 (local address 127.0.0.1:59102)
0806 20:25:43.264478 (+ 16070us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.264498 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:43.265003 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:43.265738 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:43.265747 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:43.265787 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:43.266364 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.266382 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.269074 (+ 2692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.269082 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.271741 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.271757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.273468 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.273476 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.273680 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.273734 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 20:25:43.273832 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 20:25:43.273929 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15885,"mutex_wait_us":1,"thread_start_us":15662,"threads_started":1}
I20260806 20:25:43.274353 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.248777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59102 (local address 127.17.109.253:46779)
0806 20:25:43.263859 (+ 15082us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.263868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.264631 (+ 763us) server_negotiation.cc:407] Connection header received
0806 20:25:43.265253 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.265258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.265336 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.265448 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:43.266819 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.268834 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.271931 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.273211 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.273403 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.273475 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.273872 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.274012 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":14907,"thread_start_us":14679,"threads_started":1}
I20260806 20:25:43.274434 25147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c885244b0041460db0bc5fef74135f33 in term 1.
I20260806 20:25:43.274959 24925 leader_election.cc:302] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: af3861184a084cb482576e2fb6783735, c885244b0041460db0bc5fef74135f33; no voters: f0eb02a82a7f41d4b87f3d54a1c243d5
I20260806 20:25:43.275102 24951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af3861184a084cb482576e2fb6783735" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c885244b0041460db0bc5fef74135f33" is_pre_election: true
I20260806 20:25:43.275395 25714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 20:25:43.275631 25714 raft_consensus.cc:482] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f0eb02a82a7f41d4b87f3d54a1c243d5)
I20260806 20:25:43.275774 25714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 1 FOLLOWER]: Advancing to term 2
I20260806 20:25:43.278052 25714 raft_consensus.cc:504] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } }
I20260806 20:25:43.278671 24951 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate af3861184a084cb482576e2fb6783735 in current term 2: Already voted for candidate c885244b0041460db0bc5fef74135f33 in this term.
I20260806 20:25:43.278972 25714 leader_election.cc:288] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [CANDIDATE]: Term 2 election: Requested vote from peers f0eb02a82a7f41d4b87f3d54a1c243d5 (127.17.109.254:40431), af3861184a084cb482576e2fb6783735 (127.17.109.252:42373)
I20260806 20:25:43.279486 25147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c885244b0041460db0bc5fef74135f33" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "af3861184a084cb482576e2fb6783735"
I20260806 20:25:43.279764 25147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 1 FOLLOWER]: Advancing to term 2
I20260806 20:25:43.280122 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.279038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34764)
0806 20:25:43.279879 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.280000 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":715}
I20260806 20:25:43.280311 25123 leader_election.cc:302] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: af3861184a084cb482576e2fb6783735; no voters: c885244b0041460db0bc5fef74135f33, f0eb02a82a7f41d4b87f3d54a1c243d5
I20260806 20:25:43.282776 25147 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c885244b0041460db0bc5fef74135f33 in term 2.
I20260806 20:25:43.282915 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.282517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34786)
0806 20:25:43.282684 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.282762 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 20:25:43.283027 25715 raft_consensus.cc:2726] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 20:25:43.283154 24927 leader_election.cc:334] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f0eb02a82a7f41d4b87f3d54a1c243d5 (127.17.109.254:40431): Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
I20260806 20:25:43.283322 24925 leader_election.cc:302] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: af3861184a084cb482576e2fb6783735, c885244b0041460db0bc5fef74135f33; no voters: f0eb02a82a7f41d4b87f3d54a1c243d5
I20260806 20:25:43.283639 25714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 20:25:43.284278 25714 raft_consensus.cc:686] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [term 2 LEADER]: Becoming Leader. State: Replica: c885244b0041460db0bc5fef74135f33, State: Running, Role: LEADER
I20260806 20:25:43.284749 25714 consensus_queue.cc:237] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } }
I20260806 20:25:43.286811 25725 sys_catalog.cc:455] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c885244b0041460db0bc5fef74135f33. Latest consensus state: current_term: 2 leader_uuid: "c885244b0041460db0bc5fef74135f33" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:43.287150 25725 sys_catalog.cc:458] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:43.287794 25727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:43.291721 25727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:43.292573 25727 catalog_manager.cc:1259] Loaded cluster ID: f7cb91e23f7e4e45b613e3176eff6d62
I20260806 20:25:43.292645 25727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:43.293813 25727 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:43.294611 25727 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Loaded TSK: 0
I20260806 20:25:43.295512 25727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:25:43.341115 24941 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:25:43.348248 24941 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:43.352962 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.352549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34798)
0806 20:25:43.352742 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.352821 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 20:25:43.353127 25147 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [term 2 FOLLOWER]: Refusing update from remote peer c885244b0041460db0bc5fef74135f33: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 20:25:43.354055 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.353609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34806)
0806 20:25:43.353784 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.353875 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 20:25:43.353947 25725 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 20:25:43.354270 24927 consensus_peers.cc:487] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 -> Peer f0eb02a82a7f41d4b87f3d54a1c243d5 (127.17.109.254:40431): Couldn't send request to peer f0eb02a82a7f41d4b87f3d54a1c243d5. Status: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 20:25:43.356906 25715 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c885244b0041460db0bc5fef74135f33. Latest consensus state: current_term: 2 leader_uuid: "c885244b0041460db0bc5fef74135f33" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:43.357435 25715 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:25:43.358956 25725 sys_catalog.cc:455] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "c885244b0041460db0bc5fef74135f33" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:43.359333 25725 sys_catalog.cc:458] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:43.359583 25715 sys_catalog.cc:455] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "c885244b0041460db0bc5fef74135f33" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f0eb02a82a7f41d4b87f3d54a1c243d5" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40431 } } peers { permanent_uuid: "c885244b0041460db0bc5fef74135f33" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 46779 } attrs { promote: false } } peers { permanent_uuid: "af3861184a084cb482576e2fb6783735" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 42373 } attrs { promote: false } } }
I20260806 20:25:43.359817 25729 mvcc.cc:205] Tried to move back new op lower bound from 7315652375966040064 to 7315652375698321408. Current Snapshot: MvccSnapshot[applied={T|T < 7315652375966040064}]
I20260806 20:25:43.359982 25715 sys_catalog.cc:458] T 00000000000000000000000000000000 P af3861184a084cb482576e2fb6783735 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:25:43.376044 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.368368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:46033 (local address 127.0.0.1:39698)
0806 20:25:43.368553 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.368574 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:43.368700 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:43.369784 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:43.369788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:43.369811 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:43.370063 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.370079 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.371832 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.371840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.373766 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.373781 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.374980 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.374986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.375259 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.375284 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 20:25:43.375409 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 20:25:43.375675 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":70}
I20260806 20:25:43.376896 25734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.368498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39698 (local address 127.17.109.194:46033)
0806 20:25:43.369436 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.369441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.369459 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:25:43.369529 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.369534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.369613 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.369772 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:43.370227 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.371633 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.373921 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.374805 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.374975 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.375054 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.375438 (+ 384us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.376641 (+ 1203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":739,"threads_started":1}
I20260806 20:25:43.377640 25417 tablet_service.cc:1467] Processing CreateTablet for tablet 0feb88da581248db9de9449dd2848b4e (DEFAULT_TABLE table=test-table [id=79b27c81aef9470a8256761f849f87b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:43.377952 25716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.368794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:38705 (local address 127.0.0.1:38332)
0806 20:25:43.369149 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.369170 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:43.369279 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:43.371294 (+ 2015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:43.371298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:43.371341 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:43.371642 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.371656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.373296 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.373302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.375522 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.375538 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.377200 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.377206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.377391 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.377425 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 20:25:43.377526 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 20:25:43.377669 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":38}
I20260806 20:25:43.377974 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.369506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.17.109.193:38705)
0806 20:25:43.370949 (+ 1443us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.370954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.370969 (+ 15us) server_negotiation.cc:407] Connection header received
0806 20:25:43.371071 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.371075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.371145 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.371254 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:43.371784 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.373154 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.375750 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.376968 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.377369 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.377449 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.377548 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.377684 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1326,"thread_start_us":695,"threads_started":1}
I20260806 20:25:43.378141 25417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0feb88da581248db9de9449dd2848b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:43.378772 25256 tablet_service.cc:1467] Processing CreateTablet for tablet 0feb88da581248db9de9449dd2848b4e (DEFAULT_TABLE table=test-table [id=79b27c81aef9470a8256761f849f87b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:43.379166 25256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0feb88da581248db9de9449dd2848b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:43.380966 25736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.369863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:45435 (local address 127.0.0.1:53632)
0806 20:25:43.372276 (+ 2413us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.372300 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:43.372419 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:43.372964 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:43.372968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:43.372990 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:43.373541 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.373552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.375025 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.375030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.378318 (+ 3288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.378336 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.380164 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.380171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.380352 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.380383 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 20:25:43.380471 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 20:25:43.380599 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2296,"mutex_wait_us":51,"thread_start_us":2083,"threads_started":1}
I20260806 20:25:43.381625 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.370446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.17.109.195:45435)
0806 20:25:43.370908 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.370914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.372529 (+ 1615us) server_negotiation.cc:407] Connection header received
0806 20:25:43.372608 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.372613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.372685 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.372777 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:43.373774 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.374864 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.378507 (+ 3643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.379774 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.379927 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.380007 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.381206 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.381365 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":257,"threads_started":1}
I20260806 20:25:43.382337 25567 tablet_service.cc:1467] Processing CreateTablet for tablet 0feb88da581248db9de9449dd2848b4e (DEFAULT_TABLE table=test-table [id=79b27c81aef9470a8256761f849f87b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:43.382767 25567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0feb88da581248db9de9449dd2848b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:43.384338 25738 tablet_bootstrap.cc:492] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: Bootstrap starting.
I20260806 20:25:43.386325 25738 tablet_bootstrap.cc:654] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:43.388931 25738 tablet_bootstrap.cc:492] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: No bootstrap required, opened a new log
I20260806 20:25:43.389082 25738 ts_tablet_manager.cc:1397] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 20:25:43.389736 25738 raft_consensus.cc:348] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.389938 25738 raft_consensus.cc:374] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:43.390028 25738 raft_consensus.cc:729] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a641597171fd4befae1ecec6faaf9dbc, State: Initialized, Role: FOLLOWER
I20260806 20:25:43.390264 25738 consensus_queue.cc:260] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.391095 25738 ts_tablet_manager.cc:1428] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 20:25:43.395380 25486 heartbeater.cc:503] Master 127.17.109.253:46779 was elected leader, sending a full tablet report...
I20260806 20:25:43.395956 25742 tablet_bootstrap.cc:492] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: Bootstrap starting.
I20260806 20:25:43.397718 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.391714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.194:41297)
0806 20:25:43.397042 (+ 5328us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.397298 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5138,"thread_start_us":4932,"threads_started":1}
I20260806 20:25:43.398053 25742 tablet_bootstrap.cc:654] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:43.399169 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.398619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.194:51079)
0806 20:25:43.398844 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.398969 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 20:25:43.400430 25742 tablet_bootstrap.cc:492] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: No bootstrap required, opened a new log
I20260806 20:25:43.400600 25742 ts_tablet_manager.cc:1397] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 20:25:43.401309 25742 raft_consensus.cc:348] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.401532 25742 raft_consensus.cc:374] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:43.401635 25742 raft_consensus.cc:729] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf387168244743de91a29001aad33725, State: Initialized, Role: FOLLOWER
I20260806 20:25:43.401944 25742 consensus_queue.cc:260] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.402865 25742 ts_tablet_manager.cc:1428] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 20:25:43.403228 25740 tablet_bootstrap.cc:492] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: Bootstrap starting.
I20260806 20:25:43.405360 25740 tablet_bootstrap.cc:654] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:43.405578 25636 heartbeater.cc:503] Master 127.17.109.253:46779 was elected leader, sending a full tablet report...
I20260806 20:25:43.408107 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.403408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.195:57609)
0806 20:25:43.407546 (+ 4138us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.407833 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3971,"thread_start_us":3774,"threads_started":1}
I20260806 20:25:43.409108 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.408788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.195:38425)
0806 20:25:43.408933 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.409003 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 20:25:43.409224 25740 tablet_bootstrap.cc:492] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: No bootstrap required, opened a new log
I20260806 20:25:43.409384 25740 ts_tablet_manager.cc:1397] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 20:25:43.410094 25740 raft_consensus.cc:348] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.410327 25740 raft_consensus.cc:374] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:43.410446 25740 raft_consensus.cc:729] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda73047fa6741b3b9fe04c0c6eb62fa, State: Initialized, Role: FOLLOWER
I20260806 20:25:43.410744 25740 consensus_queue.cc:260] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.411666 25740 ts_tablet_manager.cc:1428] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 20:25:43.413328 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.412186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.193:37137)
0806 20:25:43.412804 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.412991 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":307,"threads_started":1}
I20260806 20:25:43.414143 25327 heartbeater.cc:503] Master 127.17.109.253:46779 was elected leader, sending a full tablet report...
I20260806 20:25:43.414381 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.413960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.193:37011)
0806 20:25:43.414131 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.414248 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 20:25:43.628624 25746 raft_consensus.cc:482] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:43.628909 25746 raft_consensus.cc:504] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.630599 25746 leader_election.cc:288] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a641597171fd4befae1ecec6faaf9dbc (127.17.109.194:46033), eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705)
I20260806 20:25:43.636991 25734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.631105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:35871 (local address 127.17.109.194:46033)
0806 20:25:43.631330 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.631338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.631362 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:25:43.631459 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.631467 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.631557 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.631684 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:43.632273 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.633077 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.634987 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.635939 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.636127 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.636229 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.636559 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.636705 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:25:43.637028 25747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.630785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:46033 (local address 127.17.109.195:35871)
0806 20:25:43.631013 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.631034 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:43.631210 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:43.631746 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:43.631755 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:43.631804 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:43.632099 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.632109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.633222 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.633231 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.634804 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.634817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.636133 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.636140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.636375 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.636401 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 20:25:43.636529 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 20:25:43.636701 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":25}
I20260806 20:25:43.637367 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.631099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:38705 (local address 127.17.109.195:53199)
0806 20:25:43.631898 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.631913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:43.632055 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:43.632404 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:43.632408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:43.632428 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:43.632737 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.632751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.633785 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.633791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.635145 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:43.635157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.636302 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.636312 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:43.636500 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.636522 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 20:25:43.636621 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 20:25:43.636784 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":51,"thread_start_us":480,"threads_started":1}
I20260806 20:25:43.637375 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.631126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:53199 (local address 127.17.109.193:38705)
0806 20:25:43.631386 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.631396 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.632005 (+ 609us) server_negotiation.cc:407] Connection header received
0806 20:25:43.632210 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.632234 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.632293 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.632388 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:43.632902 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.633646 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.635301 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.636096 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.636273 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.636350 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.636648 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.636780 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 20:25:43.638056 25437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0feb88da581248db9de9449dd2848b4e" candidate_uuid: "cf387168244743de91a29001aad33725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a641597171fd4befae1ecec6faaf9dbc" is_pre_election: true
I20260806 20:25:43.638134 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0feb88da581248db9de9449dd2848b4e" candidate_uuid: "cf387168244743de91a29001aad33725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" is_pre_election: true
I20260806 20:25:43.638410 25276 raft_consensus.cc:2454] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf387168244743de91a29001aad33725 in term 0.
I20260806 20:25:43.638468 25437 raft_consensus.cc:2454] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf387168244743de91a29001aad33725 in term 0.
I20260806 20:25:43.638948 25514 leader_election.cc:302] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf387168244743de91a29001aad33725, eda73047fa6741b3b9fe04c0c6eb62fa; no voters:
I20260806 20:25:43.639358 25746 raft_consensus.cc:2781] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:43.639508 25746 raft_consensus.cc:482] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:43.639675 25746 raft_consensus.cc:3037] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:43.641976 25746 raft_consensus.cc:504] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.642604 25746 leader_election.cc:288] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [CANDIDATE]: Term 1 election: Requested vote from peers a641597171fd4befae1ecec6faaf9dbc (127.17.109.194:46033), eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705)
I20260806 20:25:43.643077 25437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0feb88da581248db9de9449dd2848b4e" candidate_uuid: "cf387168244743de91a29001aad33725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a641597171fd4befae1ecec6faaf9dbc"
I20260806 20:25:43.643112 25276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0feb88da581248db9de9449dd2848b4e" candidate_uuid: "cf387168244743de91a29001aad33725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa"
I20260806 20:25:43.643317 25437 raft_consensus.cc:3037] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:43.643330 25276 raft_consensus.cc:3037] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:43.646687 25276 raft_consensus.cc:2454] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf387168244743de91a29001aad33725 in term 1.
I20260806 20:25:43.646703 25437 raft_consensus.cc:2454] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf387168244743de91a29001aad33725 in term 1.
I20260806 20:25:43.647143 25514 leader_election.cc:302] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a641597171fd4befae1ecec6faaf9dbc, cf387168244743de91a29001aad33725, eda73047fa6741b3b9fe04c0c6eb62fa; no voters:
I20260806 20:25:43.647500 25746 raft_consensus.cc:2781] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:43.647998 25746 raft_consensus.cc:686] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 1 LEADER]: Becoming Leader. State: Replica: cf387168244743de91a29001aad33725, State: Running, Role: LEADER
I20260806 20:25:43.648803 25746 consensus_queue.cc:237] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } }
I20260806 20:25:43.649156 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.648639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.195:38997)
0806 20:25:43.648892 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.648989 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260806 20:25:43.652460 24941 catalog_manager.cc:5495] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf387168244743de91a29001aad33725 (127.17.109.195). New cstate: current_term: 1 leader_uuid: "cf387168244743de91a29001aad33725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf387168244743de91a29001aad33725" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 45435 } health_report { overall_health: HEALTHY } } }
I20260806 20:25:43.679301 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.673839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.17.109.195:45435)
0806 20:25:43.674066 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.674072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.674094 (+ 22us) server_negotiation.cc:407] Connection header received
0806 20:25:43.674223 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.674227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.674314 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.674435 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:43.674769 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.675475 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.676950 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.677481 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.677576 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.678047 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:43.678640 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:43.678757 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.678873 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.678948 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:25:43.683590 25276 raft_consensus.cc:1261] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 1 FOLLOWER]: Refusing update from remote peer cf387168244743de91a29001aad33725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:43.684028 25437 raft_consensus.cc:1261] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 1 FOLLOWER]: Refusing update from remote peer cf387168244743de91a29001aad33725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:43.684376 25752 consensus_queue.cc:1035] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda73047fa6741b3b9fe04c0c6eb62fa" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:43.684854 25746 consensus_queue.cc:1035] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a641597171fd4befae1ecec6faaf9dbc" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:43.687750 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.686729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.193:52997)
0806 20:25:43.686925 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.687070 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 20:25:43.688452 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.688014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.194:35377)
0806 20:25:43.688163 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.688272 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 20:25:43.692883 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.692249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.195:50807)
0806 20:25:43.692583 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.692696 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
I20260806 20:25:43.699808 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.699276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.195:42067)
0806 20:25:43.699472 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.699605 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 20:25:43.700399 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.699873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.194:57703)
0806 20:25:43.700144 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.700245 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260806 20:25:43.702546 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.695337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.17.109.193:33663)
0806 20:25:43.702267 (+ 6930us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.702378 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6812}
I20260806 20:25:43.708840 25757 mvcc.cc:205] Tried to move back new op lower bound from 7315652377321197568 to 7315652377189531648. Current Snapshot: MvccSnapshot[applied={T|T < 7315652377321197568}]
I20260806 20:25:43.711239 24941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:43.711529 24941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:43.718434 24941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c885244b0041460db0bc5fef74135f33: Sending DeleteTablet for 3 replicas of tablet 0feb88da581248db9de9449dd2848b4e
I20260806 20:25:43.719487 25417 tablet_service.cc:1514] Processing DeleteTablet for tablet 0feb88da581248db9de9449dd2848b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:43 UTC) from {username='oryx', principal='oryx/127.17.109.253@KRBTEST.COM'} at 127.0.0.1:39698
I20260806 20:25:43.719537 25256 tablet_service.cc:1514] Processing DeleteTablet for tablet 0feb88da581248db9de9449dd2848b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:43 UTC) from {username='oryx', principal='oryx/127.17.109.253@KRBTEST.COM'} at 127.0.0.1:38332
I20260806 20:25:43.720114 25567 tablet_service.cc:1514] Processing DeleteTablet for tablet 0feb88da581248db9de9449dd2848b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:43 UTC) from {username='oryx', principal='oryx/127.17.109.253@KRBTEST.COM'} at 127.0.0.1:53632
I20260806 20:25:43.720633 25766 tablet_replica.cc:330] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: stopping tablet replica
I20260806 20:25:43.720995 25766 raft_consensus.cc:2229] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:43.721446 25766 raft_consensus.cc:2258] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:43.722985 25766 ts_tablet_manager.cc:1905] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:43.723389 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 25124
I20260806 20:25:43.731473 25767 tablet_replica.cc:330] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: stopping tablet replica
I20260806 20:25:43.731812 25767 raft_consensus.cc:2229] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:43.732667 25767 raft_consensus.cc:2258] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:43.732704 25766 ts_tablet_manager.cc:1918] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:43.732980 25766 log.cc:1196] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/0feb88da581248db9de9449dd2848b4e
I20260806 20:25:43.733649 25766 ts_tablet_manager.cc:1939] T 0feb88da581248db9de9449dd2848b4e P eda73047fa6741b3b9fe04c0c6eb62fa: Deleting consensus metadata
I20260806 20:25:43.734385 25767 ts_tablet_manager.cc:1905] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:43.735260 25765 tablet_replica.cc:330] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: stopping tablet replica
I20260806 20:25:43.735291 24926 catalog_manager.cc:4841] TS eda73047fa6741b3b9fe04c0c6eb62fa (127.17.109.193:38705): tablet 0feb88da581248db9de9449dd2848b4e (table test-table [id=79b27c81aef9470a8256761f849f87b5]) successfully deleted
I20260806 20:25:43.735522 25765 raft_consensus.cc:2229] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:43.735935 25765 raft_consensus.cc:2258] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:43.737190 25765 ts_tablet_manager.cc:1905] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:43.742260 25767 ts_tablet_manager.cc:1918] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:43.742419 25767 log.cc:1196] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/0feb88da581248db9de9449dd2848b4e
I20260806 20:25:43.742959 25767 ts_tablet_manager.cc:1939] T 0feb88da581248db9de9449dd2848b4e P cf387168244743de91a29001aad33725: Deleting consensus metadata
I20260806 20:25:43.743834 25765 ts_tablet_manager.cc:1918] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:43.744247 24928 catalog_manager.cc:4841] TS cf387168244743de91a29001aad33725 (127.17.109.195:45435): tablet 0feb88da581248db9de9449dd2848b4e (table test-table [id=79b27c81aef9470a8256761f849f87b5]) successfully deleted
I20260806 20:25:43.744503 25765 log.cc:1196] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/0feb88da581248db9de9449dd2848b4e
I20260806 20:25:43.745024 25765 ts_tablet_manager.cc:1939] T 0feb88da581248db9de9449dd2848b4e P a641597171fd4befae1ecec6faaf9dbc: Deleting consensus metadata
I20260806 20:25:43.746111 24928 catalog_manager.cc:4841] TS a641597171fd4befae1ecec6faaf9dbc (127.17.109.194:46033): tablet 0feb88da581248db9de9449dd2848b4e (table test-table [id=79b27c81aef9470a8256761f849f87b5]) successfully deleted
I20260806 20:25:43.755451 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.725598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.17.109.193:38705)
0806 20:25:43.725840 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.725850 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.740819 (+ 14969us) server_negotiation.cc:407] Connection header received
0806 20:25:43.741064 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.741070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.741154 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.741269 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:43.743401 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.744390 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.745712 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.746232 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.746358 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.748972 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:43.748995 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:43.749011 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:43.749042 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:43.752710 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:43.753338 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:43.753349 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:43.753353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:43.753515 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:43.753873 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:43.753879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:43.753884 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:43.754331 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:43.754657 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.754809 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.755061 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:25:43.756390 25320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:43.808634 25736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.808118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34808)
0806 20:25:43.808331 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.808447 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 20:25:43.809921 25736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.809442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40431 (local address 127.0.0.1:34810)
0806 20:25:43.809632 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:43.809754 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:40431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 20:25:43.857077 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 25124
I20260806 20:25:43.884840 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 25337
I20260806 20:25:43.898742 25734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:43.885713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.17.109.194:46033)
0806 20:25:43.885945 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:43.885950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:43.885971 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:25:43.886165 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:43.886171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:43.886258 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:43.886396 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:43.887782 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.888533 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.889757 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:43.890154 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:43.890284 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:43.893073 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:43.893088 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:43.893105 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:43.893137 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:43.896096 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:43.896771 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:43.896778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:43.896780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:43.896899 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:43.897254 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:43.897260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:43.897264 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:43.897629 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:43.897944 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:43.898187 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 20:25:43.898418 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 20:25:43.899348 25479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:44.023382 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 25337
I20260806 20:25:44.058225 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 25497
I20260806 20:25:44.073532 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:44.059045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53638 (local address 127.17.109.195:45435)
0806 20:25:44.059246 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:44.059257 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:44.059381 (+ 124us) server_negotiation.cc:407] Connection header received
0806 20:25:44.059606 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:44.059628 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:44.059703 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:44.059805 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:44.061407 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:44.062195 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:44.063523 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:44.064201 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:44.064351 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:44.066944 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:44.066958 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:44.066973 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:44.067007 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:44.070480 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:44.071401 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:44.071408 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:44.071411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:44.071613 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:44.072037 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:44.072040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:44.072042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:44.072376 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:44.072642 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:44.073010 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 20:25:44.073184 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:25:44.074223 25628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:44.218974 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 25497
I20260806 20:25:44.244793 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 24799
I20260806 20:25:44.245908 24972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:44.372248 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 24799
I20260806 20:25:44.397406 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 24904
I20260806 20:25:44.398547 25169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:44.518263 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 24904
2026-08-06T20:25:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10528 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:44 dist-test-slave-6k22 krb5kdc[25776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:44 dist-test-slave-6k22 krb5kdc[25776](info): set up 2 sockets
Aug 06 20:25:44 dist-test-slave-6k22 krb5kdc[25776](info): commencing operation
krb5kdc: starting...
W20260806 20:25:45.737039 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:45 dist-test-slave-6k22 krb5kdc[25776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.254.keytab.
Entry for principal oryx/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.254.keytab.
I20260806 20:25:45.999783 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:32823
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.254.keytab
--principal=oryx/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:46.338238 25789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:46.338542 25789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:46.338634 25789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:46.345786 25789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:46.345889 25789 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 20:25:46.345930 25789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:46.345991 25789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:46.346041 25789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:46.346103 25789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:46.355939 25789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:32823
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.254.keytab
--principal=oryx/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:46.357156 25789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:46.358434 25789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:46.370365 25795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:46.370298 25798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:46.370541 25796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:46.371135 25789 server_base.cc:1034] running on GCE node
I20260806 20:25:46.371897 25789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:46.373230 25789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:46.374445 25789 hybrid_clock.cc:648] HybridClock initialized: now 1786047946374375 us; error 45 us; skew 500 ppm
I20260806 20:25:46.374802 25789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:46 dist-test-slave-6k22 krb5kdc[25776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047946, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:46.385064 25789 init.cc:377] Logged in from keytab as oryx/127.17.109.254@KRBTEST.COM (short username oryx)
I20260806 20:25:46.386596 25789 webserver.cc:460] Webserver started at http://127.17.109.254:33263/ using document root <none> and password file <none>
I20260806 20:25:46.387326 25789 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:46.387441 25789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:46.387809 25789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:46.390872 25789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "56e216e598bc4b6f8f14fcc40ff54c78"
format_stamp: "Formatted at 2026-08-06 20:25:46 on dist-test-slave-6k22"
server_key: "60259d7699eb115a2430ecff57856dea"
server_key_iv: "0f3b207a318519884d68590b7f18e998"
server_key_version: "encryptionkey@0"
I20260806 20:25:46.391762 25789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "56e216e598bc4b6f8f14fcc40ff54c78"
format_stamp: "Formatted at 2026-08-06 20:25:46 on dist-test-slave-6k22"
server_key: "60259d7699eb115a2430ecff57856dea"
server_key_iv: "0f3b207a318519884d68590b7f18e998"
server_key_version: "encryptionkey@0"
I20260806 20:25:46.397523 25789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 20:25:46.401417 25805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:46.402812 25789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:25:46.402947 25789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "56e216e598bc4b6f8f14fcc40ff54c78"
format_stamp: "Formatted at 2026-08-06 20:25:46 on dist-test-slave-6k22"
server_key: "60259d7699eb115a2430ecff57856dea"
server_key_iv: "0f3b207a318519884d68590b7f18e998"
server_key_version: "encryptionkey@0"
I20260806 20:25:46.403119 25789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:46.457871 25789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:46.462781 25789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:46.463037 25789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:46.486325 25789 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:32823
I20260806 20:25:46.486444 25872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:32823 every 8 connection(s)
I20260806 20:25:46.487771 25789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:46.491540 25873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:46.496186 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 25789
I20260806 20:25:46.496446 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:46.497274 17847 external_mini_cluster.cc:1428] Setting key 4a0fb75cb3c13b700e1ac6d57daf47c0
I20260806 20:25:46.503783 25873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: Bootstrap starting.
I20260806 20:25:46.507548 25873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:46.508744 25873 log.cc:824] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:46.512207 25873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: No bootstrap required, opened a new log
Aug 06 20:25:46 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:46.520311 25873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e216e598bc4b6f8f14fcc40ff54c78" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 32823 } }
I20260806 20:25:46.520744 25873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:46.520888 25873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e216e598bc4b6f8f14fcc40ff54c78, State: Initialized, Role: FOLLOWER
I20260806 20:25:46.521446 25873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e216e598bc4b6f8f14fcc40ff54c78" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 32823 } }
I20260806 20:25:46.521697 25873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:46.521844 25873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:46.522017 25873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:46.522186 25876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:46.499456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36462 (local address 127.17.109.254:32823)
0806 20:25:46.500348 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:46.500360 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:46.500403 (+ 43us) server_negotiation.cc:407] Connection header received
0806 20:25:46.502204 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:46.502226 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:46.502606 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:46.502937 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:46.504817 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:46.506315 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:46.508175 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:46.508972 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:46.509121 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:46.514145 (+ 5024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:46.514154 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:46.514168 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:46.514206 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:46.517666 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:46.518413 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:46.518420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:46.518425 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:46.518605 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:46.519028 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:46.519032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:46.519034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:46.519641 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:46.519943 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:46.520376 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 20:25:46.520822 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":392,"threads_started":1}
I20260806 20:25:46.525024 25873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e216e598bc4b6f8f14fcc40ff54c78" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 32823 } }
I20260806 20:25:46.525578 25873 leader_election.cc:302] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56e216e598bc4b6f8f14fcc40ff54c78; no voters:
I20260806 20:25:46.526155 25873 leader_election.cc:288] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:46.526362 25878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:46.527194 25878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [term 1 LEADER]: Becoming Leader. State: Replica: 56e216e598bc4b6f8f14fcc40ff54c78, State: Running, Role: LEADER
I20260806 20:25:46.527827 25878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e216e598bc4b6f8f14fcc40ff54c78" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 32823 } }
I20260806 20:25:46.528342 25873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:46.530586 25879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56e216e598bc4b6f8f14fcc40ff54c78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e216e598bc4b6f8f14fcc40ff54c78" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 32823 } } }
I20260806 20:25:46.531049 25879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:46.531643 25880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56e216e598bc4b6f8f14fcc40ff54c78. Latest consensus state: current_term: 1 leader_uuid: "56e216e598bc4b6f8f14fcc40ff54c78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e216e598bc4b6f8f14fcc40ff54c78" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 32823 } } }
I20260806 20:25:46.531950 25880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:46.534216 25885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:46.540364 25885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:46.549866 25885 catalog_manager.cc:1347] Generated new cluster ID: 12573e4377c445249d83868cb6f0a1d7
I20260806 20:25:46.550024 25885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:46.561549 25885 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:46.562913 25885 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:46.574752 25885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: Generated new TSK 0
I20260806 20:25:46.575731 25885 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab.
Entry for principal oryx/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab.
I20260806 20:25:46.651474 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:32823
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab
--principal=oryx/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:46.996807 25900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:46.997107 25900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:46.997222 25900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:47.005337 25900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:47.005456 25900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:47.005541 25900 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:25:47.005623 25900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:47.015578 25900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab
--principal=oryx/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:47.016799 25900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:47.018078 25900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:47.031575 25908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:47.031528 25905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:47.031697 25906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:47.032686 25900 server_base.cc:1034] running on GCE node
I20260806 20:25:47.033290 25900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:47.034243 25900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:47.035451 25900 hybrid_clock.cc:648] HybridClock initialized: now 1786047947035392 us; error 34 us; skew 500 ppm
I20260806 20:25:47.035775 25900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:47 dist-test-slave-6k22 krb5kdc[25776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047947, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:47.039036 25900 init.cc:377] Logged in from keytab as oryx/127.17.109.193@KRBTEST.COM (short username oryx)
I20260806 20:25:47.040584 25900 webserver.cc:460] Webserver started at http://127.17.109.193:44429/ using document root <none> and password file <none>
I20260806 20:25:47.041404 25900 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:47.041530 25900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:47.041867 25900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:47.044816 25900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "8eb4881f92bc43f9be696e27659442ea"
format_stamp: "Formatted at 2026-08-06 20:25:47 on dist-test-slave-6k22"
server_key: "58409d75dc632c1812298fbf04c047ae"
server_key_iv: "5a5df210c30376b24991cbdea34b4e66"
server_key_version: "encryptionkey@0"
I20260806 20:25:47.045603 25900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "8eb4881f92bc43f9be696e27659442ea"
format_stamp: "Formatted at 2026-08-06 20:25:47 on dist-test-slave-6k22"
server_key: "58409d75dc632c1812298fbf04c047ae"
server_key_iv: "5a5df210c30376b24991cbdea34b4e66"
server_key_version: "encryptionkey@0"
I20260806 20:25:47.050637 25900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 20:25:47.054479 25915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:47.056083 25900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:47.056339 25900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "8eb4881f92bc43f9be696e27659442ea"
format_stamp: "Formatted at 2026-08-06 20:25:47 on dist-test-slave-6k22"
server_key: "58409d75dc632c1812298fbf04c047ae"
server_key_iv: "5a5df210c30376b24991cbdea34b4e66"
server_key_version: "encryptionkey@0"
I20260806 20:25:47.056612 25900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:47.087203 25900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:47.091643 25900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:47.091930 25900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:47.094719 25900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:47.094842 25900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:47.094992 25900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:47.095081 25900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:25:47 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047947, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:47.124599 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:47.095649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:41929 (local address 127.17.109.254:32823)
0806 20:25:47.096227 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:47.096235 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:47.097554 (+ 1319us) server_negotiation.cc:407] Connection header received
0806 20:25:47.100836 (+ 3282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:47.100846 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:47.100971 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:47.101096 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:47.103865 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.104936 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.108351 (+ 3415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.108899 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.109008 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:47.115547 (+ 6539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:47.115588 (+ 41us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:47.115595 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:47.115642 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:47.119655 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:47.121023 (+ 1368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:47.121032 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:47.121037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:47.121206 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:47.121919 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:47.121927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:47.121933 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:47.122370 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:47.122546 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:47.123836 (+ 1290us) server_negotiation.cc:299] Negotiation successful
0806 20:25:47.124118 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":322,"threads_started":1}
I20260806 20:25:47.126904 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:47.096368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:32823 (local address 127.17.109.193:41929)
0806 20:25:47.097355 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:47.097420 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:47.100434 (+ 3014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:47.101427 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:47.101443 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:47.102461 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:47.103624 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:47.103659 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.105139 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.105148 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:47.108146 (+ 2998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:47.108166 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.109190 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.109202 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:47.109683 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:47.110646 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:47.110700 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:47.115223 (+ 4523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:47.119930 (+ 4707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:47.119944 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:47.119970 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:47.120851 (+ 881us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:47.121353 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:47.121361 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:47.121368 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:47.121764 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:47.122553 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:47.122567 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:47.123223 (+ 656us) client_negotiation.cc:769] Sending connection context
0806 20:25:47.123690 (+ 467us) client_negotiation.cc:240] Negotiation successful
0806 20:25:47.124522 (+ 832us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":366,"threads_started":1}
I20260806 20:25:47.141600 25900 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:38741
I20260806 20:25:47.141561 26044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:38741 every 8 connection(s)
I20260806 20:25:47.143167 25900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:47.145259 26045 heartbeater.cc:347] Connected to a master server at 127.17.109.254:32823
I20260806 20:25:47.146343 26045 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:47.147054 26045 heartbeater.cc:511] Master 127.17.109.254:32823 requested a full tablet report, sending...
I20260806 20:25:47.148785 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:47.133797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:42809 (local address 127.17.109.254:32823)
0806 20:25:47.133978 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:47.133986 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:47.134165 (+ 179us) server_negotiation.cc:407] Connection header received
0806 20:25:47.134367 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:47.134372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:47.134443 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:47.134539 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:47.136032 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.137078 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.139064 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.139765 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.139890 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:47.142457 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:47.142462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:47.142465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:47.142494 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:47.145710 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:47.146537 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:47.146543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:47.146552 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:47.146685 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:47.147099 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:47.147107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:47.147111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:47.147443 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:47.147653 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:47.148123 (+ 470us) server_negotiation.cc:299] Negotiation successful
0806 20:25:47.148370 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:25:47.148834 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:47.133428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:32823 (local address 127.17.109.193:42809)
0806 20:25:47.134030 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:47.134052 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:47.134205 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:47.134578 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:47.134583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:47.135201 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:47.135818 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:47.135835 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.137226 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.137249 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:47.138921 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:47.138940 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.139957 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.139964 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:47.140118 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:47.140872 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:47.140912 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:47.142281 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:47.145856 (+ 3575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:47.145862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:47.145869 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:47.146396 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:47.146806 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:47.146812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:47.146816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:47.146978 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:47.147607 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:47.147612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:47.147848 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 20:25:47.148016 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 20:25:47.148345 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":566,"threads_started":1}
I20260806 20:25:47.149206 25824 ts_manager.cc:194] Registered new tserver with Master: 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741)
I20260806 20:25:47.150628 25824 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:41929
I20260806 20:25:47.153563 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 25900
I20260806 20:25:47.153813 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:47.154458 17847 external_mini_cluster.cc:1428] Setting key 726ab75ff64906323803a5952eea6d84
WARNING: no policy specified for oryx/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab.
Entry for principal oryx/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab.
I20260806 20:25:47.210610 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:32823
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab
--principal=oryx/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:47.558869 26049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:47.559190 26049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:47.559360 26049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:47.566941 26049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:47.567085 26049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:47.567215 26049 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:25:47.567306 26049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:47.577647 26049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab
--principal=oryx/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:47.579010 26049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:47.580412 26049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:47.593909 26054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:47.593889 26057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:47.594209 26055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:47.594727 26049 server_base.cc:1034] running on GCE node
I20260806 20:25:47.595400 26049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:47.596375 26049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:47.597625 26049 hybrid_clock.cc:648] HybridClock initialized: now 1786047947597536 us; error 63 us; skew 500 ppm
I20260806 20:25:47.597959 26049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:47 dist-test-slave-6k22 krb5kdc[25776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047947, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:47.601310 26049 init.cc:377] Logged in from keytab as oryx/127.17.109.194@KRBTEST.COM (short username oryx)
I20260806 20:25:47.602895 26049 webserver.cc:460] Webserver started at http://127.17.109.194:41167/ using document root <none> and password file <none>
I20260806 20:25:47.603688 26049 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:47.603818 26049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:47.604142 26049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:47.607158 26049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "5fc7387b9219407387e9f80b87e44a51"
format_stamp: "Formatted at 2026-08-06 20:25:47 on dist-test-slave-6k22"
server_key: "a0c312da744d4efed733ebf1c8c02879"
server_key_iv: "70e89d656d2bd174a678962f0949f9a8"
server_key_version: "encryptionkey@0"
I20260806 20:25:47.608019 26049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "5fc7387b9219407387e9f80b87e44a51"
format_stamp: "Formatted at 2026-08-06 20:25:47 on dist-test-slave-6k22"
server_key: "a0c312da744d4efed733ebf1c8c02879"
server_key_iv: "70e89d656d2bd174a678962f0949f9a8"
server_key_version: "encryptionkey@0"
I20260806 20:25:47.613159 26049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:25:47.616963 26064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:47.618417 26049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:25:47.618549 26049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "5fc7387b9219407387e9f80b87e44a51"
format_stamp: "Formatted at 2026-08-06 20:25:47 on dist-test-slave-6k22"
server_key: "a0c312da744d4efed733ebf1c8c02879"
server_key_iv: "70e89d656d2bd174a678962f0949f9a8"
server_key_version: "encryptionkey@0"
I20260806 20:25:47.618737 26049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:47.641408 26049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:47.645555 26049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:47.645787 26049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:47.648519 26049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:47.648600 26049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:47.648764 26049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:47.648850 26049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:25:47 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047947, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:47.674692 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:47.648991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:49459 (local address 127.17.109.254:32823)
0806 20:25:47.650027 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:47.650033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:47.650862 (+ 829us) server_negotiation.cc:407] Connection header received
0806 20:25:47.652935 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:47.652942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:47.653043 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:47.653197 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:47.655429 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.656889 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.660444 (+ 3555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.661123 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.661288 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:47.666360 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:47.666366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:47.666370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:47.666410 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:47.670595 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:47.671429 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:47.671438 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:47.671444 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:47.671666 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:47.672462 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:47.672469 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:47.672473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:47.672809 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:47.673004 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:47.673857 (+ 853us) server_negotiation.cc:299] Negotiation successful
0806 20:25:47.674149 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":240,"server-negotiator.queue_time_us":879,"thread_start_us":512,"threads_started":1}
I20260806 20:25:47.676040 26074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:47.649720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:32823 (local address 127.17.109.194:49459)
0806 20:25:47.650672 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:47.650720 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:47.652650 (+ 1930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:47.653538 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:47.653549 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:47.654438 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:47.655235 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:47.655254 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.657120 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.657128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:47.660206 (+ 3078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:47.660225 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.661457 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.661466 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:47.661806 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:47.662613 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:47.662657 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:47.666093 (+ 3436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:47.670759 (+ 4666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:47.670767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:47.670781 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:47.671270 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:47.671848 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:47.671857 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:47.671863 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:47.672302 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:47.672976 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:47.672985 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:47.673403 (+ 418us) client_negotiation.cc:769] Sending connection context
0806 20:25:47.673719 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 20:25:47.674356 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":393,"threads_started":1}
I20260806 20:25:47.701499 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:47.682981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:32823 (local address 127.17.109.194:53959)
0806 20:25:47.683948 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:47.683968 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:47.684146 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:47.684569 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:47.684578 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:47.685334 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:47.685832 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:47.685850 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.686935 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.686943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:47.689127 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:47.689150 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.690160 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.690170 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:47.690459 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:47.691205 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:47.691236 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:47.692698 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:47.697396 (+ 4698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:47.697405 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:47.697411 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:47.698164 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:47.698610 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:47.698618 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:47.698624 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:47.698838 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:47.700009 (+ 1171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:47.700014 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:47.700332 (+ 318us) client_negotiation.cc:769] Sending connection context
0806 20:25:47.700528 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 20:25:47.700892 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":676,"threads_started":1}
I20260806 20:25:47.702001 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:47.682923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:53959 (local address 127.17.109.254:32823)
0806 20:25:47.683138 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:47.683143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:47.684063 (+ 920us) server_negotiation.cc:407] Connection header received
0806 20:25:47.684296 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:47.684302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:47.684395 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:47.684510 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:47.686029 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.686794 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.689374 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:47.689927 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:47.690079 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:47.692913 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:47.692918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:47.692922 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:47.692954 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:47.697164 (+ 4210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:47.698329 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:47.698334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:47.698336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:47.698490 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:47.699009 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:47.699013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:47.699016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:47.699851 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:47.700047 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:47.701530 (+ 1483us) server_negotiation.cc:299] Negotiation successful
0806 20:25:47.701752 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 20:25:47.713608 26049 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:46855
I20260806 20:25:47.713698 26189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:46855 every 8 connection(s)
I20260806 20:25:47.714867 26049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:47.717092 26190 heartbeater.cc:347] Connected to a master server at 127.17.109.254:32823
I20260806 20:25:47.717296 26190 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:47.717850 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 26049
I20260806 20:25:47.717893 26190 heartbeater.cc:511] Master 127.17.109.254:32823 requested a full tablet report, sending...
I20260806 20:25:47.718060 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:47.718492 17847 external_mini_cluster.cc:1428] Setting key 8ae938f05e6764d4fd19c1dbe2ea0253
I20260806 20:25:47.719455 25823 ts_manager.cc:194] Registered new tserver with Master: 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194:46855)
I20260806 20:25:47.720603 25823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:49459
WARNING: no policy specified for oryx/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab.
Entry for principal oryx/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab.
I20260806 20:25:47.776744 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:32823
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab
--principal=oryx/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:48.121968 26194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:48.122290 26194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:48.122452 26194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:48.130115 26194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:48.130251 26194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:48.130403 26194 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 20:25:48.130540 26194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:48.140556 26194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab
--principal=oryx/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.109.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:48.141948 26194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:48.143249 26194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 20:25:48.153764 26045 heartbeater.cc:503] Master 127.17.109.254:32823 was elected leader, sending a full tablet report...
W20260806 20:25:48.156229 26199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:48.156539 26200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:48.156821 26202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:48.157796 26194 server_base.cc:1034] running on GCE node
I20260806 20:25:48.158490 26194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:48.159502 26194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:48.160791 26194 hybrid_clock.cc:648] HybridClock initialized: now 1786047948160701 us; error 55 us; skew 500 ppm
I20260806 20:25:48.161226 26194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:48 dist-test-slave-6k22 krb5kdc[25776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047948, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:48.164572 26194 init.cc:377] Logged in from keytab as oryx/127.17.109.195@KRBTEST.COM (short username oryx)
I20260806 20:25:48.166077 26194 webserver.cc:460] Webserver started at http://127.17.109.195:38057/ using document root <none> and password file <none>
I20260806 20:25:48.166811 26194 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:48.166932 26194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:48.167290 26194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:48.170431 26194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "093d3938c8c14c2c86aeb5b60f838197"
format_stamp: "Formatted at 2026-08-06 20:25:48 on dist-test-slave-6k22"
server_key: "77dabdb45ee6f2ce2c0c7204783345cd"
server_key_iv: "ddee2c01506a35e427351ccceffa02d9"
server_key_version: "encryptionkey@0"
I20260806 20:25:48.171309 26194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "093d3938c8c14c2c86aeb5b60f838197"
format_stamp: "Formatted at 2026-08-06 20:25:48 on dist-test-slave-6k22"
server_key: "77dabdb45ee6f2ce2c0c7204783345cd"
server_key_iv: "ddee2c01506a35e427351ccceffa02d9"
server_key_version: "encryptionkey@0"
I20260806 20:25:48.177395 26194 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260806 20:25:48.183020 26209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:48.184626 26194 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 20:25:48.184780 26194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "093d3938c8c14c2c86aeb5b60f838197"
format_stamp: "Formatted at 2026-08-06 20:25:48 on dist-test-slave-6k22"
server_key: "77dabdb45ee6f2ce2c0c7204783345cd"
server_key_iv: "ddee2c01506a35e427351ccceffa02d9"
server_key_version: "encryptionkey@0"
I20260806 20:25:48.184991 26194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:48.212431 26194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:48.216984 26194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:48.217234 26194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:48.220151 26194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:48.220273 26194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:48.220417 26194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:48.220498 26194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:25:48 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047948, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:48.244168 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.220884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:36563 (local address 127.17.109.254:32823)
0806 20:25:48.221367 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.221377 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.222794 (+ 1417us) server_negotiation.cc:407] Connection header received
0806 20:25:48.225057 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.225063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.225149 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.225283 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:48.227629 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.228765 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.231520 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.232202 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.232340 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.237407 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:48.237414 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:48.237418 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:48.237462 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:48.241029 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:48.241743 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:48.241749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:48.241752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:48.241892 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:48.242335 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:48.242340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:48.242344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:48.242613 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:48.242768 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.243507 (+ 739us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.243760 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":246,"threads_started":1}
I20260806 20:25:48.245639 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.221761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:32823 (local address 127.17.109.195:36563)
0806 20:25:48.222603 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:48.222640 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:48.224770 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:48.225535 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:48.225543 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:48.226427 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:48.227365 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.227392 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.228925 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.228932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.231318 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.231336 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.232487 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.232497 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.232855 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.233610 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:48.233651 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:48.237105 (+ 3454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:48.241167 (+ 4062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:48.241176 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:48.241195 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:48.241617 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:48.242009 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:48.242013 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:48.242017 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:48.242219 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:48.242764 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:48.242772 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:48.243130 (+ 358us) client_negotiation.cc:769] Sending connection context
0806 20:25:48.243403 (+ 273us) client_negotiation.cc:240] Negotiation successful
0806 20:25:48.243980 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":394,"threads_started":1}
I20260806 20:25:48.269358 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.251865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:39611 (local address 127.17.109.254:32823)
0806 20:25:48.252043 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.252051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.253043 (+ 992us) server_negotiation.cc:407] Connection header received
0806 20:25:48.253255 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.253261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.253331 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.253429 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:48.254818 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.256279 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.258426 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.259090 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.259219 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.261793 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:48.261802 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:48.261809 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:48.261855 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:48.265550 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:48.266731 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:48.266738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:48.266744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:48.266920 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:48.267375 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:48.267382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:48.267385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:48.267821 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:48.268004 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.268666 (+ 662us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.268942 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 20:25:48.269917 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.251970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:32823 (local address 127.17.109.195:39611)
0806 20:25:48.252938 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:48.252954 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:48.253116 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:48.253461 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:48.253466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:48.254145 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:48.254558 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.254568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.256467 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.256475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.258278 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.258290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.259299 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.259305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.259474 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.260490 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:48.260520 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:48.261592 (+ 1072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:48.265727 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:48.265733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:48.265740 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:48.266547 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:48.267077 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:48.267087 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:48.267091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:48.267263 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:48.267981 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:48.267987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:48.268281 (+ 294us) client_negotiation.cc:769] Sending connection context
0806 20:25:48.268562 (+ 281us) client_negotiation.cc:240] Negotiation successful
0806 20:25:48.269093 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":606,"threads_started":1}
I20260806 20:25:48.272243 26194 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:33139
I20260806 20:25:48.272279 26342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:33139 every 8 connection(s)
I20260806 20:25:48.273871 26194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:48.274858 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 26194
I20260806 20:25:48.275080 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:48.275705 17847 external_mini_cluster.cc:1428] Setting key 5df0979e74ccd8e40626582e52196fe7
I20260806 20:25:48.276019 26343 heartbeater.cc:347] Connected to a master server at 127.17.109.254:32823
I20260806 20:25:48.276247 26343 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:48.276934 26343 heartbeater.cc:511] Master 127.17.109.254:32823 requested a full tablet report, sending...
I20260806 20:25:48.278326 25824 ts_manager.cc:194] Registered new tserver with Master: 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:48.279138 25824 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:36563
I20260806 20:25:48.291321 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 20:25:48 dist-test-slave-6k22 krb5kdc[25776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047948, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 20:25:48 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:48 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 20:25:48.352494 26218 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:48.341355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36464 (local address 127.17.109.254:32823)
0806 20:25:48.341601 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.341608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.341866 (+ 258us) server_negotiation.cc:407] Connection header received
0806 20:25:48.342023 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.342027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.342106 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.342194 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:48.343679 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.344398 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.345642 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.346049 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.346140 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.352241 (+ 6101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36464: BlockingRecv error: recv got EOF from 127.0.0.1:36464 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 20:25:48 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047948, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:48.371011 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.355926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36478 (local address 127.17.109.254:32823)
0806 20:25:48.356147 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.356155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.356467 (+ 312us) server_negotiation.cc:407] Connection header received
0806 20:25:48.356691 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.356695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.356767 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.356879 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:48.358126 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.358914 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.360598 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.361069 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.361215 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.365290 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:48.365295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:48.365298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:48.365333 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:48.368137 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:48.368873 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:48.368878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:48.368881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:48.369019 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:48.369442 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:48.369448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:48.369452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:48.369764 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:48.370067 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.370406 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.370657 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 20:25:48.375409 25824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:25:48.383482 25824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:48.409140 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.397216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.17.109.194:46855)
0806 20:25:48.400706 (+ 3490us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.400728 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.400766 (+ 38us) server_negotiation.cc:407] Connection header received
0806 20:25:48.400892 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.400904 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.401237 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.401466 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:48.402336 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.403808 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.406793 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.407982 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.408186 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.408383 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.408664 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.408840 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3279,"thread_start_us":3152,"threads_started":1}
I20260806 20:25:48.409266 26361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.396893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:46855 (local address 127.0.0.1:56752)
0806 20:25:48.398947 (+ 2054us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:48.398980 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:48.399122 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:48.401738 (+ 2616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:48.401746 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:48.401779 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:48.402145 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.402160 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.403991 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.403996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.406560 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.406578 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.408209 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.408224 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.408426 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.408475 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 20:25:48.408619 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 20:25:48.408820 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1879,"mutex_wait_us":1,"thread_start_us":1672,"threads_started":1}
I20260806 20:25:48.411315 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.398799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.17.109.195:33139)
0806 20:25:48.400174 (+ 1375us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.400187 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.401509 (+ 1322us) server_negotiation.cc:407] Connection header received
0806 20:25:48.401826 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.401836 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.402100 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.402307 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:48.403363 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.405235 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.408246 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.409917 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.410130 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.410299 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.410881 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.411055 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1170,"thread_start_us":1005,"threads_started":1}
I20260806 20:25:48.411877 26366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.398092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33139 (local address 127.0.0.1:42036)
0806 20:25:48.401361 (+ 3269us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:48.401391 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:48.401624 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:48.402544 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:48.402551 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:48.402585 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:48.403086 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.403098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.405431 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.405440 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.408023 (+ 2583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.408040 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.410458 (+ 2418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.410465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.410657 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.410712 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 20:25:48.410819 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 20:25:48.410966 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3135,"mutex_wait_us":1,"thread_start_us":2974,"threads_started":1}
I20260806 20:25:48.412755 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.397371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54480 (local address 127.17.109.193:38741)
0806 20:25:48.401752 (+ 4381us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.401762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.401793 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:25:48.401871 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.401876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.402066 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.402272 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:48.403139 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.404958 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.409587 (+ 4629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.411287 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.411494 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.411688 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.412308 (+ 620us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.412475 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4192,"thread_start_us":4029,"threads_started":1}
I20260806 20:25:48.412765 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.396838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:38741 (local address 127.0.0.1:54480)
0806 20:25:48.397826 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:48.397905 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:48.398149 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:48.402260 (+ 4111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:48.402265 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:48.402292 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:48.402855 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.402876 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.405147 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.405155 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.409332 (+ 4177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.409352 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.411870 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.411877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.412075 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.412111 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 20:25:48.412248 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 20:25:48.412381 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":1188,"thread_start_us":712,"threads_started":1}
I20260806 20:25:48.414413 26116 tablet_service.cc:1467] Processing CreateTablet for tablet 736314686c9145c8b6d96b90676c2780 (DEFAULT_TABLE table=test-table [id=7bc268bdacf3485391cc3e832f9516bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:48.415235 26269 tablet_service.cc:1467] Processing CreateTablet for tablet 736314686c9145c8b6d96b90676c2780 (DEFAULT_TABLE table=test-table [id=7bc268bdacf3485391cc3e832f9516bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:48.416070 26116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736314686c9145c8b6d96b90676c2780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:48.416603 26269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736314686c9145c8b6d96b90676c2780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:48.416536 25969 tablet_service.cc:1467] Processing CreateTablet for tablet 736314686c9145c8b6d96b90676c2780 (DEFAULT_TABLE table=test-table [id=7bc268bdacf3485391cc3e832f9516bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:48.417796 25969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 736314686c9145c8b6d96b90676c2780. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:48.429797 26369 tablet_bootstrap.cc:492] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Bootstrap starting.
I20260806 20:25:48.431893 26370 tablet_bootstrap.cc:492] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Bootstrap starting.
I20260806 20:25:48.434505 26369 tablet_bootstrap.cc:654] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:48.435931 26370 tablet_bootstrap.cc:654] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:48.436020 26369 log.cc:824] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:48.437556 26370 log.cc:824] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:48.439201 26371 tablet_bootstrap.cc:492] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: Bootstrap starting.
I20260806 20:25:48.442081 26369 tablet_bootstrap.cc:492] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: No bootstrap required, opened a new log
I20260806 20:25:48.442431 26369 ts_tablet_manager.cc:1397] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260806 20:25:48.442513 26370 tablet_bootstrap.cc:492] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: No bootstrap required, opened a new log
I20260806 20:25:48.442875 26370 ts_tablet_manager.cc:1397] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260806 20:25:48.444145 26371 tablet_bootstrap.cc:654] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:48.445608 26371 log.cc:824] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:48.449529 26371 tablet_bootstrap.cc:492] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: No bootstrap required, opened a new log
I20260806 20:25:48.449851 26371 ts_tablet_manager.cc:1397] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 20:25:48.450917 26369 raft_consensus.cc:348] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.451290 26369 raft_consensus.cc:374] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:48.451412 26369 raft_consensus.cc:729] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Initialized, Role: FOLLOWER
I20260806 20:25:48.452039 26369 consensus_queue.cc:260] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.455694 26370 raft_consensus.cc:348] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.456218 26370 raft_consensus.cc:374] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:48.456346 26370 raft_consensus.cc:729] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093d3938c8c14c2c86aeb5b60f838197, State: Initialized, Role: FOLLOWER
I20260806 20:25:48.457036 26370 consensus_queue.cc:260] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.458431 26343 heartbeater.cc:503] Master 127.17.109.254:32823 was elected leader, sending a full tablet report...
I20260806 20:25:48.459180 26370 ts_tablet_manager.cc:1428] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260806 20:25:48.461789 26371 raft_consensus.cc:348] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.462188 26371 raft_consensus.cc:374] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:48.462318 26371 raft_consensus.cc:729] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb4881f92bc43f9be696e27659442ea, State: Initialized, Role: FOLLOWER
I20260806 20:25:48.462952 26371 consensus_queue.cc:260] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.464236 26190 heartbeater.cc:503] Master 127.17.109.254:32823 was elected leader, sending a full tablet report...
I20260806 20:25:48.464910 26369 ts_tablet_manager.cc:1428] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Time spent starting tablet: real 0.022s user 0.014s sys 0.007s
I20260806 20:25:48.466977 26371 ts_tablet_manager.cc:1428] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
W20260806 20:25:48.467113 26191 tablet.cc:2369] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:25:48.525612 26344 tablet.cc:2369] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:25:48.528625 26376 raft_consensus.cc:482] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:48.528903 26376 raft_consensus.cc:504] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.530558 26376 leader_election.cc:288] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:48.537820 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.531125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:58779 (local address 127.17.109.193:38741)
0806 20:25:48.531350 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.531356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.532193 (+ 837us) server_negotiation.cc:407] Connection header received
0806 20:25:48.532440 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.532446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.532528 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.532623 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:48.533181 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.533896 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.535685 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.536648 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.537011 (+ 363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.537097 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.537422 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.537551 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 20:25:48.537957 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.531169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:38741 (local address 127.17.109.194:58779)
0806 20:25:48.532080 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:48.532099 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:48.532272 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:48.532650 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:48.532654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:48.532689 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:48.533045 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.533054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.534025 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.534031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.535510 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.535522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.537070 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.537077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.537234 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.537258 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 20:25:48.537395 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 20:25:48.537543 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":51,"thread_start_us":594,"threads_started":1}
I20260806 20:25:48.538902 26367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.531234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:36241 (local address 127.17.109.195:33139)
0806 20:25:48.531445 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.531450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.532764 (+ 1314us) server_negotiation.cc:407] Connection header received
0806 20:25:48.533041 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.533053 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.533129 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.533247 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:25:48.533922 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.534709 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.536953 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.537983 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.538135 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.538216 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.538596 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.538712 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:25:48.539049 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.531703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33139 (local address 127.17.109.194:36241)
0806 20:25:48.532628 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:48.532647 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:48.532857 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:48.533265 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:48.533271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:48.533298 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:25:48.533726 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.533744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.534879 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.534886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.536637 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:48.536657 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.538243 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.538252 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:48.538437 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.538462 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 20:25:48.538537 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 20:25:48.538658 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":6,"thread_start_us":510,"threads_started":1}
I20260806 20:25:48.539194 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736314686c9145c8b6d96b90676c2780" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea" is_pre_election: true
I20260806 20:25:48.539846 25990 raft_consensus.cc:2454] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:48.540194 26295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736314686c9145c8b6d96b90676c2780" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197" is_pre_election: true
I20260806 20:25:48.540547 26068 leader_election.cc:302] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc7387b9219407387e9f80b87e44a51, 8eb4881f92bc43f9be696e27659442ea; no voters:
I20260806 20:25:48.540771 26295 raft_consensus.cc:2454] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:48.540928 26376 raft_consensus.cc:2781] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:48.541072 26376 raft_consensus.cc:482] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:48.541183 26376 raft_consensus.cc:3037] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:48.543375 26376 raft_consensus.cc:504] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.544034 26376 leader_election.cc:288] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:48.544425 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736314686c9145c8b6d96b90676c2780" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea"
I20260806 20:25:48.544552 26295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "736314686c9145c8b6d96b90676c2780" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197"
I20260806 20:25:48.544607 25990 raft_consensus.cc:3037] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:48.544768 26295 raft_consensus.cc:3037] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:48.546875 25990 raft_consensus.cc:2454] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:48.546911 26295 raft_consensus.cc:2454] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:48.547260 26068 leader_election.cc:302] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc7387b9219407387e9f80b87e44a51, 8eb4881f92bc43f9be696e27659442ea; no voters:
I20260806 20:25:48.547529 26376 raft_consensus.cc:2781] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:48.548089 26376 raft_consensus.cc:686] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Running, Role: LEADER
I20260806 20:25:48.548637 26376 consensus_queue.cc:237] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.553131 25824 catalog_manager.cc:5495] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } health_report { overall_health: UNKNOWN } } }
I20260806 20:25:48.569710 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:48.564567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46976 (local address 127.17.109.194:46855)
0806 20:25:48.564754 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:48.564758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:48.564775 (+ 17us) server_negotiation.cc:407] Connection header received
0806 20:25:48.564989 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:48.564992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:48.565070 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:48.565193 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:48.565509 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.566248 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.567609 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:48.568001 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:48.568095 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:48.568515 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:48.569143 (+ 628us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:48.569288 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:48.569340 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 20:25:48.569412 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 20:25:48.577819 25990 raft_consensus.cc:1261] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:48.577847 26295 raft_consensus.cc:1261] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:48.578925 26381 consensus_queue.cc:1035] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:48.579332 26376 consensus_queue.cc:1035] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:48.590880 26384 mvcc.cc:205] Tried to move back new op lower bound from 7315652397364117504 to 7315652397259321344. Current Snapshot: MvccSnapshot[applied={T|T < 7315652397364117504}]
I20260806 20:25:48.599726 26385 mvcc.cc:205] Tried to move back new op lower bound from 7315652397364117504 to 7315652397259321344. Current Snapshot: MvccSnapshot[applied={T|T < 7315652397364117504 or (T in {7315652397364117504})}]
I20260806 20:25:48.605491 25822 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:48.605820 25822 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:48.610136 25822 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: Sending DeleteTablet for 3 replicas of tablet 736314686c9145c8b6d96b90676c2780
I20260806 20:25:48.611488 26116 tablet_service.cc:1514] Processing DeleteTablet for tablet 736314686c9145c8b6d96b90676c2780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:48 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:56752
I20260806 20:25:48.611488 25969 tablet_service.cc:1514] Processing DeleteTablet for tablet 736314686c9145c8b6d96b90676c2780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:48 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:54480
I20260806 20:25:48.611661 26269 tablet_service.cc:1514] Processing DeleteTablet for tablet 736314686c9145c8b6d96b90676c2780 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:48 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:42036
I20260806 20:25:48.614516 25822 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:25:48.616677 25822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:48.624262 26394 tablet_replica.cc:330] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: stopping tablet replica
I20260806 20:25:48.625099 26394 raft_consensus.cc:2229] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:48.625804 26394 raft_consensus.cc:2258] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:48.626881 26395 tablet_replica.cc:330] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: stopping tablet replica
I20260806 20:25:48.630100 26394 ts_tablet_manager.cc:1905] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:48.632561 26115 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8770a1748445a08521df440aaa7f12 (DEFAULT_TABLE table=test-table [id=5e1e61dbcc04481eae1cdb126d64a1a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:48.629629 26395 raft_consensus.cc:2229] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:48.632860 25969 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8770a1748445a08521df440aaa7f12 (DEFAULT_TABLE table=test-table [id=5e1e61dbcc04481eae1cdb126d64a1a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:48.633350 26115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8770a1748445a08521df440aaa7f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:48.633384 25969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8770a1748445a08521df440aaa7f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:48.633718 26395 raft_consensus.cc:2258] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:48.633929 26393 tablet_replica.cc:330] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: stopping tablet replica
I20260806 20:25:48.634352 26393 raft_consensus.cc:2229] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:48.635053 26393 raft_consensus.cc:2258] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:48.636884 26395 ts_tablet_manager.cc:1905] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:48.640777 26369 tablet_bootstrap.cc:492] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: Bootstrap starting.
I20260806 20:25:48.642704 26369 tablet_bootstrap.cc:654] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:48.643690 26269 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8770a1748445a08521df440aaa7f12 (DEFAULT_TABLE table=test-table [id=5e1e61dbcc04481eae1cdb126d64a1a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:48.644248 26269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8770a1748445a08521df440aaa7f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:48.649781 26393 ts_tablet_manager.cc:1905] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:48.649919 26371 tablet_bootstrap.cc:492] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: Bootstrap starting.
I20260806 20:25:48.652102 26371 tablet_bootstrap.cc:654] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:48.659641 26395 ts_tablet_manager.cc:1918] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:48.659910 26395 log.cc:1196] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/736314686c9145c8b6d96b90676c2780
I20260806 20:25:48.660545 26394 ts_tablet_manager.cc:1918] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:48.663718 26394 log.cc:1196] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/736314686c9145c8b6d96b90676c2780
I20260806 20:25:48.665445 26394 ts_tablet_manager.cc:1939] T 736314686c9145c8b6d96b90676c2780 P 8eb4881f92bc43f9be696e27659442ea: Deleting consensus metadata
I20260806 20:25:48.665870 26393 ts_tablet_manager.cc:1918] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:48.666067 26393 log.cc:1196] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/736314686c9145c8b6d96b90676c2780
I20260806 20:25:48.666610 26369 tablet_bootstrap.cc:492] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: No bootstrap required, opened a new log
I20260806 20:25:48.666927 26370 tablet_bootstrap.cc:492] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: Bootstrap starting.
I20260806 20:25:48.668609 26395 ts_tablet_manager.cc:1939] T 736314686c9145c8b6d96b90676c2780 P 093d3938c8c14c2c86aeb5b60f838197: Deleting consensus metadata
I20260806 20:25:48.669147 26370 tablet_bootstrap.cc:654] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:48.669409 26371 tablet_bootstrap.cc:492] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: No bootstrap required, opened a new log
I20260806 20:25:48.669802 26371 ts_tablet_manager.cc:1397] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.000s
I20260806 20:25:48.670785 25807 catalog_manager.cc:4841] TS 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139): tablet 736314686c9145c8b6d96b90676c2780 (table test-table [id=7bc268bdacf3485391cc3e832f9516bb]) successfully deleted
I20260806 20:25:48.671041 26371 raft_consensus.cc:348] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.671582 26371 raft_consensus.cc:374] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:48.671790 26371 raft_consensus.cc:729] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb4881f92bc43f9be696e27659442ea, State: Initialized, Role: FOLLOWER
I20260806 20:25:48.666656 26393 ts_tablet_manager.cc:1939] T 736314686c9145c8b6d96b90676c2780 P 5fc7387b9219407387e9f80b87e44a51: Deleting consensus metadata
I20260806 20:25:48.672410 26371 consensus_queue.cc:260] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.673192 26371 ts_tablet_manager.cc:1428] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 20:25:48.673949 25808 catalog_manager.cc:4841] TS 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194:46855): tablet 736314686c9145c8b6d96b90676c2780 (table test-table [id=7bc268bdacf3485391cc3e832f9516bb]) successfully deleted
I20260806 20:25:48.675594 26369 ts_tablet_manager.cc:1397] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.017s
I20260806 20:25:48.676615 26369 raft_consensus.cc:348] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.678113 25809 catalog_manager.cc:4841] TS 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741): tablet 736314686c9145c8b6d96b90676c2780 (table test-table [id=7bc268bdacf3485391cc3e832f9516bb]) successfully deleted
I20260806 20:25:48.678485 26369 raft_consensus.cc:374] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:48.678675 26369 raft_consensus.cc:729] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Initialized, Role: FOLLOWER
I20260806 20:25:48.679126 26369 consensus_queue.cc:260] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.680030 26369 ts_tablet_manager.cc:1428] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 20:25:48.681464 26370 tablet_bootstrap.cc:492] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: No bootstrap required, opened a new log
I20260806 20:25:48.681659 26370 ts_tablet_manager.cc:1397] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260806 20:25:48.683121 26370 raft_consensus.cc:348] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.683466 26370 raft_consensus.cc:374] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:48.683868 26370 raft_consensus.cc:729] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093d3938c8c14c2c86aeb5b60f838197, State: Initialized, Role: FOLLOWER
I20260806 20:25:48.684294 26370 consensus_queue.cc:260] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:48.684988 26370 ts_tablet_manager.cc:1428] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260806 20:25:48.896349 26046 tablet.cc:2369] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:25:49.058660 26381 raft_consensus.cc:482] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:49.058984 26381 raft_consensus.cc:504] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.059835 26381 leader_election.cc:288] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:49.060554 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8770a1748445a08521df440aaa7f12" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea" is_pre_election: true
I20260806 20:25:49.060578 26295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8770a1748445a08521df440aaa7f12" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197" is_pre_election: true
I20260806 20:25:49.060842 25990 raft_consensus.cc:2454] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:49.060897 26295 raft_consensus.cc:2454] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:49.061431 26066 leader_election.cc:302] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093d3938c8c14c2c86aeb5b60f838197, 5fc7387b9219407387e9f80b87e44a51; no voters:
I20260806 20:25:49.061883 26381 raft_consensus.cc:2781] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:49.062038 26381 raft_consensus.cc:482] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:49.062264 26381 raft_consensus.cc:3037] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.064813 26381 raft_consensus.cc:504] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.065516 26381 leader_election.cc:288] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:49.066118 26295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8770a1748445a08521df440aaa7f12" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197"
I20260806 20:25:49.066079 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8770a1748445a08521df440aaa7f12" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea"
I20260806 20:25:49.066365 26295 raft_consensus.cc:3037] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.066365 25990 raft_consensus.cc:3037] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.069756 26295 raft_consensus.cc:2454] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:49.069761 25990 raft_consensus.cc:2454] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:49.070286 26068 leader_election.cc:302] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc7387b9219407387e9f80b87e44a51, 8eb4881f92bc43f9be696e27659442ea; no voters:
I20260806 20:25:49.070603 26381 raft_consensus.cc:2781] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:49.070817 26381 raft_consensus.cc:686] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Running, Role: LEADER
I20260806 20:25:49.071139 26381 consensus_queue.cc:237] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.074318 25822 catalog_manager.cc:5495] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } health_report { overall_health: UNKNOWN } } }
Aug 06 20:25:49 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047946, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.254@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:49.151592 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.134882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.17.109.254:32823)
0806 20:25:49.135730 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:49.135739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:49.136118 (+ 379us) server_negotiation.cc:407] Connection header received
0806 20:25:49.136296 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:49.136303 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:49.136401 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:49.136536 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:49.138386 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.139686 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.141119 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.141769 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.141922 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.145637 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:49.145642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:49.145645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:49.145671 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:49.148827 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:49.149575 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:49.149583 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:49.149587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:49.149722 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:49.150082 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:49.150089 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:49.150092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:49.150347 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:49.150515 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:49.150897 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 20:25:49.151219 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":645,"thread_start_us":3497,"threads_started":1}
I20260806 20:25:49.151592 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.134888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:32823 (local address 127.0.0.1:40422)
0806 20:25:49.135983 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:49.136003 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:49.136161 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:49.136632 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:49.136661 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:49.137916 (+ 1255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:49.138211 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:49.138223 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.139828 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.139834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:49.140978 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:49.141002 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.142164 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.142170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:49.142373 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.143124 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:49.143142 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:49.145439 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:49.148943 (+ 3504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:49.148957 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:49.148965 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:49.149462 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:49.149833 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:49.149837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:49.149841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:49.149984 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:49.150479 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:49.150488 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:49.150636 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 20:25:49.150804 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 20:25:49.151087 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"thread_start_us":719,"threads_started":1}
I20260806 20:25:49.158254 25822 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:40422:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 20:25:49.159592 25822 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:49.166929 25969 tablet_service.cc:1467] Processing CreateTablet for tablet fc5725e88e42465a98760902d56a0357 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2fda6ae46f84dcba1156cce758e9e13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:25:49.167208 26115 tablet_service.cc:1467] Processing CreateTablet for tablet fc5725e88e42465a98760902d56a0357 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2fda6ae46f84dcba1156cce758e9e13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:25:49.167583 25969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc5725e88e42465a98760902d56a0357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.167359 26269 tablet_service.cc:1467] Processing CreateTablet for tablet fc5725e88e42465a98760902d56a0357 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a2fda6ae46f84dcba1156cce758e9e13]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 20:25:49.167760 26115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc5725e88e42465a98760902d56a0357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.167944 26269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc5725e88e42465a98760902d56a0357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.174412 26369 tablet_bootstrap.cc:492] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: Bootstrap starting.
I20260806 20:25:49.175249 26370 tablet_bootstrap.cc:492] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: Bootstrap starting.
I20260806 20:25:49.176244 26407 tablet_bootstrap.cc:492] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: Bootstrap starting.
I20260806 20:25:49.176303 26369 tablet_bootstrap.cc:654] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.177152 26370 tablet_bootstrap.cc:654] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.177706 26407 tablet_bootstrap.cc:654] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.178478 26369 tablet_bootstrap.cc:492] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: No bootstrap required, opened a new log
I20260806 20:25:49.178709 26369 ts_tablet_manager.cc:1397] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260806 20:25:49.179759 26369 raft_consensus.cc:348] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.180023 26369 raft_consensus.cc:374] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.180110 26369 raft_consensus.cc:729] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.180143 26407 tablet_bootstrap.cc:492] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: No bootstrap required, opened a new log
I20260806 20:25:49.180287 26407 ts_tablet_manager.cc:1397] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 20:25:49.180406 26369 consensus_queue.cc:260] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.180946 26370 tablet_bootstrap.cc:492] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: No bootstrap required, opened a new log
I20260806 20:25:49.180845 26381 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.181125 26381 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:25:49.181131 26370 ts_tablet_manager.cc:1397] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260806 20:25:49.181005 26407 raft_consensus.cc:348] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.181231 26407 raft_consensus.cc:374] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.181324 26407 raft_consensus.cc:729] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb4881f92bc43f9be696e27659442ea, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.181567 26407 consensus_queue.cc:260] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.181941 26370 raft_consensus.cc:348] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.182193 26370 raft_consensus.cc:374] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.182276 26370 raft_consensus.cc:729] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093d3938c8c14c2c86aeb5b60f838197, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.182598 26370 consensus_queue.cc:260] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.182857 26369 ts_tablet_manager.cc:1428] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260806 20:25:49.183235 26377 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.183593 26377 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:25:49.184124 26407 ts_tablet_manager.cc:1428] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: Time spent starting tablet: real 0.004s user 0.001s sys 0.004s
I20260806 20:25:49.184057 26411 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.184403 26411 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:25:49.185130 26370 ts_tablet_manager.cc:1428] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260806 20:25:49.190590 26381 raft_consensus.cc:482] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:49.190793 26381 raft_consensus.cc:504] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.191473 26381 leader_election.cc:288] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:49.191960 26295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc5725e88e42465a98760902d56a0357" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197" is_pre_election: true
I20260806 20:25:49.192157 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc5725e88e42465a98760902d56a0357" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea" is_pre_election: true
I20260806 20:25:49.192227 26295 raft_consensus.cc:2454] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:49.192405 25990 raft_consensus.cc:2454] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:49.192606 26066 leader_election.cc:302] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093d3938c8c14c2c86aeb5b60f838197, 5fc7387b9219407387e9f80b87e44a51; no voters:
I20260806 20:25:49.192864 26381 raft_consensus.cc:2781] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:49.192991 26381 raft_consensus.cc:482] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:49.193107 26381 raft_consensus.cc:3037] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.195217 26381 raft_consensus.cc:504] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.195819 26381 leader_election.cc:288] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:49.196259 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc5725e88e42465a98760902d56a0357" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea"
I20260806 20:25:49.196295 26295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc5725e88e42465a98760902d56a0357" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197"
I20260806 20:25:49.196470 25990 raft_consensus.cc:3037] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.196506 26295 raft_consensus.cc:3037] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.198793 25990 raft_consensus.cc:2454] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:49.198952 26295 raft_consensus.cc:2454] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:49.199157 26068 leader_election.cc:302] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc7387b9219407387e9f80b87e44a51, 8eb4881f92bc43f9be696e27659442ea; no voters:
I20260806 20:25:49.199517 26381 raft_consensus.cc:2781] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:49.199746 26381 raft_consensus.cc:686] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Running, Role: LEADER
I20260806 20:25:49.200022 26381 consensus_queue.cc:237] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.201617 26376 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: TxnStatusTablet state changed. Reason: New leader 5fc7387b9219407387e9f80b87e44a51. Latest consensus state: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.201849 26376 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:25:49.202483 26413 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 20:25:49.202715 26413 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 20:25:49.202975 25823 catalog_manager.cc:5495] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } health_report { overall_health: UNKNOWN } } }
I20260806 20:25:49.229048 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.221874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:46855 (local address 127.0.0.1:46988)
0806 20:25:49.222274 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:49.222298 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:49.222468 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:49.222898 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:49.222904 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:49.222919 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:25:49.223140 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:49.223153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.224356 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.224363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:49.226190 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:49.226207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.227412 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.227419 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:49.227739 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.227818 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:25:49.228332 (+ 514us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:25:49.228356 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 20:25:49.228474 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 20:25:49.228648 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260806 20:25:49.230074 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.221804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.17.109.194:46855)
0806 20:25:49.222493 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:49.222499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:49.222523 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:25:49.222647 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:49.222650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:49.222745 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:49.222834 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:49.223303 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.224213 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.226359 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.227227 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.227370 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.228029 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:49.228185 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:49.228302 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:49.229541 (+ 1239us) server_negotiation.cc:299] Negotiation successful
0806 20:25:49.229626 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":385,"threads_started":1}
I20260806 20:25:49.234634 25990 raft_consensus.cc:1261] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:49.234740 26295 raft_consensus.cc:1261] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:49.235281 26381 consensus_queue.cc:1035] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:49.235649 26376 consensus_queue.cc:1035] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:49.240182 26377 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: TxnStatusTablet state changed. Reason: New leader 5fc7387b9219407387e9f80b87e44a51. Latest consensus state: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.240509 26377 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:25:49.243863 26419 mvcc.cc:205] Tried to move back new op lower bound from 7315652400058535936 to 7315652399925620736. Current Snapshot: MvccSnapshot[applied={T|T < 7315652400058535936}]
I20260806 20:25:49.244508 26381 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.244894 26381 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:25:49.250882 26377 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.251168 26377 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 093d3938c8c14c2c86aeb5b60f838197: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:25:49.252169 26411 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: TxnStatusTablet state changed. Reason: New leader 5fc7387b9219407387e9f80b87e44a51. Latest consensus state: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.252501 26411 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:25:49.254292 26376 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.254582 26376 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 5fc7387b9219407387e9f80b87e44a51: This TxnStatusTablet replica's current role is: LEADER
I20260806 20:25:49.258761 26411 tablet_replica.cc:439] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } } }
I20260806 20:25:49.259043 26411 tablet_replica.cc:442] T fc5725e88e42465a98760902d56a0357 P 8eb4881f92bc43f9be696e27659442ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 20:25:49.269474 26428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.262860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:46855 (local address 127.17.109.194:54643)
0806 20:25:49.263544 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:49.263597 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:49.263732 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:49.264070 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:49.264073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:49.264082 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 20:25:49.264382 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:49.264390 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.265969 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.265973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:49.267270 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:49.267282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.268323 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.268328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:49.268505 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.268555 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 20:25:49.268971 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 20:25:49.268979 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 20:25:49.269037 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 20:25:49.269135 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":34,"thread_start_us":340,"threads_started":1}
I20260806 20:25:49.269520 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.262822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:54643 (local address 127.17.109.194:46855)
0806 20:25:49.263014 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:49.263022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:49.263687 (+ 665us) server_negotiation.cc:407] Connection header received
0806 20:25:49.263853 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:49.263861 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:49.263947 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:49.264057 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:49.264520 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.265842 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.267409 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.268166 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.268308 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.268728 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:49.268867 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:49.268976 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:49.269061 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 20:25:49.269151 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":65}
I20260806 20:25:49.281116 26294 raft_consensus.cc:1261] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:49.281088 25990 raft_consensus.cc:1261] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:49.281744 26381 consensus_queue.cc:1035] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:49.282065 26376 consensus_queue.cc:1035] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:49.367323 25824 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:49.367652 25824 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:49.370939 25824 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: Sending DeleteTablet for 3 replicas of tablet 0f8770a1748445a08521df440aaa7f12
I20260806 20:25:49.371835 25969 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8770a1748445a08521df440aaa7f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:54480
I20260806 20:25:49.372072 26269 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8770a1748445a08521df440aaa7f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:42036
I20260806 20:25:49.372109 26115 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8770a1748445a08521df440aaa7f12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:56752
I20260806 20:25:49.384884 26434 tablet_replica.cc:330] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: stopping tablet replica
I20260806 20:25:49.385414 26434 raft_consensus.cc:2229] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:49.385964 26434 raft_consensus.cc:2258] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.386925 26435 tablet_replica.cc:330] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: stopping tablet replica
I20260806 20:25:49.387348 26435 raft_consensus.cc:2229] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:49.387451 26434 ts_tablet_manager.cc:1905] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.388131 26435 raft_consensus.cc:2258] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.390081 26435 ts_tablet_manager.cc:1905] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.395073 26434 ts_tablet_manager.cc:1918] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:49.395229 26434 log.cc:1196] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/0f8770a1748445a08521df440aaa7f12
I20260806 20:25:49.395742 26434 ts_tablet_manager.cc:1939] T 0f8770a1748445a08521df440aaa7f12 P 093d3938c8c14c2c86aeb5b60f838197: Deleting consensus metadata
I20260806 20:25:49.401574 26433 tablet_replica.cc:330] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: stopping tablet replica
I20260806 20:25:49.401795 26433 raft_consensus.cc:2229] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:49.402079 26433 raft_consensus.cc:2258] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.402972 25807 catalog_manager.cc:4841] TS 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139): tablet 0f8770a1748445a08521df440aaa7f12 (table test-table [id=5e1e61dbcc04481eae1cdb126d64a1a2]) successfully deleted
I20260806 20:25:49.404129 26435 ts_tablet_manager.cc:1918] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:49.404392 26435 log.cc:1196] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/0f8770a1748445a08521df440aaa7f12
I20260806 20:25:49.405197 26435 ts_tablet_manager.cc:1939] T 0f8770a1748445a08521df440aaa7f12 P 5fc7387b9219407387e9f80b87e44a51: Deleting consensus metadata
I20260806 20:25:49.406404 26433 ts_tablet_manager.cc:1905] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.406586 25808 catalog_manager.cc:4841] TS 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194:46855): tablet 0f8770a1748445a08521df440aaa7f12 (table test-table [id=5e1e61dbcc04481eae1cdb126d64a1a2]) successfully deleted
I20260806 20:25:49.418673 26433 ts_tablet_manager.cc:1918] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:49.418871 26433 log.cc:1196] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/0f8770a1748445a08521df440aaa7f12
I20260806 20:25:49.419512 26433 ts_tablet_manager.cc:1939] T 0f8770a1748445a08521df440aaa7f12 P 8eb4881f92bc43f9be696e27659442ea: Deleting consensus metadata
I20260806 20:25:49.420881 25809 catalog_manager.cc:4841] TS 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741): tablet 0f8770a1748445a08521df440aaa7f12 (table test-table [id=5e1e61dbcc04481eae1cdb126d64a1a2]) successfully deleted
Aug 06 20:25:49 dist-test-slave-6k22 krb5kdc[25776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 20:25:49 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:49 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 20:25:49.469120 26405 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:49.457424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.17.109.254:32823)
0806 20:25:49.457649 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:49.457657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:49.458042 (+ 385us) server_negotiation.cc:407] Connection header received
0806 20:25:49.458246 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:49.458253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:49.458346 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:49.458490 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:49.460087 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.461399 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.462762 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.463481 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.463652 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.468829 (+ 5177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40432: BlockingRecv error: recv got EOF from 127.0.0.1:40432 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 20:25:49 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM
I20260806 20:25:49.489226 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.472372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.17.109.254:32823)
0806 20:25:49.472561 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:49.472568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:49.472832 (+ 264us) server_negotiation.cc:407] Connection header received
0806 20:25:49.472968 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:49.472974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:49.473055 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:49.473170 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:49.474320 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.475615 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.476859 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.477628 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.477773 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.482113 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:49.482121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:49.482127 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:49.482185 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:49.485873 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:49.486711 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:49.486717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:49.486722 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:49.486916 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:49.487333 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:49.487341 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:49.487345 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:49.487805 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:49.488109 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:49.488454 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 20:25:49.488732 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 20:25:49.493343 25824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:25:49.494081 25824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:49.502121 25969 tablet_service.cc:1467] Processing CreateTablet for tablet 893e5ccfde8b4933a886bf1c1804e530 (DEFAULT_TABLE table=test-table [id=7456185a11dc43559e8b1399c7be0ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:49.502579 25969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893e5ccfde8b4933a886bf1c1804e530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.504065 26269 tablet_service.cc:1467] Processing CreateTablet for tablet 893e5ccfde8b4933a886bf1c1804e530 (DEFAULT_TABLE table=test-table [id=7456185a11dc43559e8b1399c7be0ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:49.504522 26269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893e5ccfde8b4933a886bf1c1804e530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.507997 26115 tablet_service.cc:1467] Processing CreateTablet for tablet 893e5ccfde8b4933a886bf1c1804e530 (DEFAULT_TABLE table=test-table [id=7456185a11dc43559e8b1399c7be0ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:49.508539 26115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 893e5ccfde8b4933a886bf1c1804e530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.509300 26407 tablet_bootstrap.cc:492] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: Bootstrap starting.
I20260806 20:25:49.510844 26407 tablet_bootstrap.cc:654] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.514765 26407 tablet_bootstrap.cc:492] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: No bootstrap required, opened a new log
I20260806 20:25:49.514943 26407 ts_tablet_manager.cc:1397] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 20:25:49.515779 26407 raft_consensus.cc:348] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.516052 26407 raft_consensus.cc:374] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.516127 26407 raft_consensus.cc:729] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb4881f92bc43f9be696e27659442ea, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.516448 26407 consensus_queue.cc:260] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.517903 26407 ts_tablet_manager.cc:1428] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 20:25:49.518471 26369 tablet_bootstrap.cc:492] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: Bootstrap starting.
I20260806 20:25:49.520263 26369 tablet_bootstrap.cc:654] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.520323 26370 tablet_bootstrap.cc:492] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: Bootstrap starting.
I20260806 20:25:49.522133 26370 tablet_bootstrap.cc:654] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.531759 26369 tablet_bootstrap.cc:492] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: No bootstrap required, opened a new log
I20260806 20:25:49.531960 26369 ts_tablet_manager.cc:1397] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260806 20:25:49.532065 26370 tablet_bootstrap.cc:492] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: No bootstrap required, opened a new log
I20260806 20:25:49.532251 26370 ts_tablet_manager.cc:1397] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.008s
I20260806 20:25:49.532855 26369 raft_consensus.cc:348] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.533119 26369 raft_consensus.cc:374] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.533253 26369 raft_consensus.cc:729] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.533111 26370 raft_consensus.cc:348] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.533429 26370 raft_consensus.cc:374] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.533525 26370 raft_consensus.cc:729] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093d3938c8c14c2c86aeb5b60f838197, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.533560 26369 consensus_queue.cc:260] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.533764 26370 consensus_queue.cc:260] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.534293 26369 ts_tablet_manager.cc:1428] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260806 20:25:49.534377 26370 ts_tablet_manager.cc:1428] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 20:25:49.605130 26376 raft_consensus.cc:482] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:49.605358 26376 raft_consensus.cc:504] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.606208 26376 leader_election.cc:288] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:49.606845 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893e5ccfde8b4933a886bf1c1804e530" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea" is_pre_election: true
I20260806 20:25:49.606870 26294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893e5ccfde8b4933a886bf1c1804e530" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197" is_pre_election: true
I20260806 20:25:49.607124 25990 raft_consensus.cc:2454] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:49.607167 26294 raft_consensus.cc:2454] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:49.607722 26068 leader_election.cc:302] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc7387b9219407387e9f80b87e44a51, 8eb4881f92bc43f9be696e27659442ea; no voters:
I20260806 20:25:49.608086 26376 raft_consensus.cc:2781] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:49.608261 26376 raft_consensus.cc:482] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:49.608376 26376 raft_consensus.cc:3037] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.612107 26376 raft_consensus.cc:504] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.612944 26376 leader_election.cc:288] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:49.613571 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893e5ccfde8b4933a886bf1c1804e530" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea"
I20260806 20:25:49.613564 26294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "893e5ccfde8b4933a886bf1c1804e530" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197"
I20260806 20:25:49.613828 26294 raft_consensus.cc:3037] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.613826 25990 raft_consensus.cc:3037] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.616513 25990 raft_consensus.cc:2454] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:49.616993 26068 leader_election.cc:302] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fc7387b9219407387e9f80b87e44a51, 8eb4881f92bc43f9be696e27659442ea; no voters:
I20260806 20:25:49.617240 26294 raft_consensus.cc:2454] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:49.617449 26376 raft_consensus.cc:2781] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:49.617671 26376 raft_consensus.cc:686] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Running, Role: LEADER
I20260806 20:25:49.618034 26376 consensus_queue.cc:237] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.621294 25824 catalog_manager.cc:5495] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } health_report { overall_health: UNKNOWN } } }
I20260806 20:25:49.652806 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.646616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47002 (local address 127.17.109.194:46855)
0806 20:25:49.646833 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:49.646837 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:49.646895 (+ 58us) server_negotiation.cc:407] Connection header received
0806 20:25:49.647131 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:49.647135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:49.647200 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:49.647288 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:25:49.647783 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.648761 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.650895 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.651325 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.651424 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.652062 (+ 638us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:25:49.652176 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:25:49.652257 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:49.652416 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 20:25:49.652477 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 20:25:49.655887 26294 raft_consensus.cc:1261] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:49.655992 25990 raft_consensus.cc:1261] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:49.656595 26376 consensus_queue.cc:1035] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:49.656901 26431 consensus_queue.cc:1035] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:49.670990 25823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:49.671262 25823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:49.674283 25823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: Sending DeleteTablet for 3 replicas of tablet 893e5ccfde8b4933a886bf1c1804e530
I20260806 20:25:49.675159 26115 tablet_service.cc:1514] Processing DeleteTablet for tablet 893e5ccfde8b4933a886bf1c1804e530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:56752
I20260806 20:25:49.675220 25969 tablet_service.cc:1514] Processing DeleteTablet for tablet 893e5ccfde8b4933a886bf1c1804e530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:54480
I20260806 20:25:49.675474 26433 tablet_replica.cc:330] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: stopping tablet replica
I20260806 20:25:49.675484 26435 tablet_replica.cc:330] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: stopping tablet replica
I20260806 20:25:49.675715 26433 raft_consensus.cc:2229] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:49.675781 26435 raft_consensus.cc:2229] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:49.675949 26433 raft_consensus.cc:2258] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.676100 26435 raft_consensus.cc:2258] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.676393 25823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 20:25:49.676692 26269 tablet_service.cc:1514] Processing DeleteTablet for tablet 893e5ccfde8b4933a886bf1c1804e530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:42036
I20260806 20:25:49.676970 26434 tablet_replica.cc:330] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: stopping tablet replica
W20260806 20:25:49.677098 25823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:25:49.677202 26434 raft_consensus.cc:2229] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:49.677286 26435 ts_tablet_manager.cc:1905] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.677465 26434 raft_consensus.cc:2258] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.679903 26434 ts_tablet_manager.cc:1905] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.685873 25969 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae272bc58bc463cac53223143ced5e2 (DEFAULT_TABLE table=test-table [id=b43a21392f094ff9b332235cb5541696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:49.685992 26269 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae272bc58bc463cac53223143ced5e2 (DEFAULT_TABLE table=test-table [id=b43a21392f094ff9b332235cb5541696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:49.686362 25969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae272bc58bc463cac53223143ced5e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.686410 26269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae272bc58bc463cac53223143ced5e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.687063 26434 ts_tablet_manager.cc:1918] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:49.687331 26434 log.cc:1196] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/893e5ccfde8b4933a886bf1c1804e530
I20260806 20:25:49.687852 26434 ts_tablet_manager.cc:1939] T 893e5ccfde8b4933a886bf1c1804e530 P 093d3938c8c14c2c86aeb5b60f838197: Deleting consensus metadata
I20260806 20:25:49.688309 26115 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae272bc58bc463cac53223143ced5e2 (DEFAULT_TABLE table=test-table [id=b43a21392f094ff9b332235cb5541696]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:25:49.688874 26115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae272bc58bc463cac53223143ced5e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:49.689836 26433 ts_tablet_manager.cc:1905] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.691912 25807 catalog_manager.cc:4841] TS 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139): tablet 893e5ccfde8b4933a886bf1c1804e530 (table test-table [id=7456185a11dc43559e8b1399c7be0ce3]) successfully deleted
I20260806 20:25:49.696625 26433 ts_tablet_manager.cc:1918] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:49.696832 26433 log.cc:1196] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/893e5ccfde8b4933a886bf1c1804e530
I20260806 20:25:49.697356 26433 ts_tablet_manager.cc:1939] T 893e5ccfde8b4933a886bf1c1804e530 P 8eb4881f92bc43f9be696e27659442ea: Deleting consensus metadata
I20260806 20:25:49.698266 26370 tablet_bootstrap.cc:492] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: Bootstrap starting.
I20260806 20:25:49.699869 25809 catalog_manager.cc:4841] TS 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741): tablet 893e5ccfde8b4933a886bf1c1804e530 (table test-table [id=7456185a11dc43559e8b1399c7be0ce3]) successfully deleted
I20260806 20:25:49.700361 26407 tablet_bootstrap.cc:492] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: Bootstrap starting.
I20260806 20:25:49.700918 26435 ts_tablet_manager.cc:1918] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:25:49.701079 26435 log.cc:1196] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/893e5ccfde8b4933a886bf1c1804e530
I20260806 20:25:49.701577 26435 ts_tablet_manager.cc:1939] T 893e5ccfde8b4933a886bf1c1804e530 P 5fc7387b9219407387e9f80b87e44a51: Deleting consensus metadata
I20260806 20:25:49.702101 26407 tablet_bootstrap.cc:654] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.702739 26369 tablet_bootstrap.cc:492] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: Bootstrap starting.
I20260806 20:25:49.702704 25808 catalog_manager.cc:4841] TS 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194:46855): tablet 893e5ccfde8b4933a886bf1c1804e530 (table test-table [id=7456185a11dc43559e8b1399c7be0ce3]) successfully deleted
I20260806 20:25:49.703825 26370 tablet_bootstrap.cc:654] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.704254 26369 tablet_bootstrap.cc:654] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:49.706565 26407 tablet_bootstrap.cc:492] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: No bootstrap required, opened a new log
I20260806 20:25:49.706729 26407 ts_tablet_manager.cc:1397] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260806 20:25:49.707602 26407 raft_consensus.cc:348] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.707898 26407 raft_consensus.cc:374] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.707993 26407 raft_consensus.cc:729] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb4881f92bc43f9be696e27659442ea, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.708230 26407 consensus_queue.cc:260] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.711829 26407 ts_tablet_manager.cc:1428] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260806 20:25:49.713341 26369 tablet_bootstrap.cc:492] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: No bootstrap required, opened a new log
I20260806 20:25:49.713536 26369 ts_tablet_manager.cc:1397] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260806 20:25:49.713840 26370 tablet_bootstrap.cc:492] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: No bootstrap required, opened a new log
I20260806 20:25:49.714051 26370 ts_tablet_manager.cc:1397] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.003s
I20260806 20:25:49.714389 26369 raft_consensus.cc:348] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.714612 26369 raft_consensus.cc:374] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.714715 26369 raft_consensus.cc:729] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.714965 26369 consensus_queue.cc:260] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.714979 26370 raft_consensus.cc:348] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.715268 26370 raft_consensus.cc:374] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:49.715360 26370 raft_consensus.cc:729] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093d3938c8c14c2c86aeb5b60f838197, State: Initialized, Role: FOLLOWER
I20260806 20:25:49.716013 26369 ts_tablet_manager.cc:1428] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260806 20:25:49.716672 26370 consensus_queue.cc:260] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.717413 26370 ts_tablet_manager.cc:1428] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 20:25:49.817953 26376 raft_consensus.cc:482] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:25:49.818150 26376 raft_consensus.cc:504] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.818827 26376 leader_election.cc:288] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:49.819393 26294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae272bc58bc463cac53223143ced5e2" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197" is_pre_election: true
I20260806 20:25:49.819478 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae272bc58bc463cac53223143ced5e2" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea" is_pre_election: true
I20260806 20:25:49.819727 26294 raft_consensus.cc:2454] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:49.819808 25990 raft_consensus.cc:2454] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 0.
I20260806 20:25:49.820251 26066 leader_election.cc:302] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093d3938c8c14c2c86aeb5b60f838197, 5fc7387b9219407387e9f80b87e44a51; no voters:
I20260806 20:25:49.820529 26376 raft_consensus.cc:2781] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:25:49.820640 26376 raft_consensus.cc:482] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:25:49.820783 26376 raft_consensus.cc:3037] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.823107 26376 raft_consensus.cc:504] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.823770 26376 leader_election.cc:288] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741), 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139)
I20260806 20:25:49.824165 25990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae272bc58bc463cac53223143ced5e2" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb4881f92bc43f9be696e27659442ea"
I20260806 20:25:49.824265 26294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae272bc58bc463cac53223143ced5e2" candidate_uuid: "5fc7387b9219407387e9f80b87e44a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093d3938c8c14c2c86aeb5b60f838197"
I20260806 20:25:49.824360 25990 raft_consensus.cc:3037] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.824514 26294 raft_consensus.cc:3037] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:49.826534 26294 raft_consensus.cc:2454] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:49.826597 25990 raft_consensus.cc:2454] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fc7387b9219407387e9f80b87e44a51 in term 1.
I20260806 20:25:49.826984 26066 leader_election.cc:302] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093d3938c8c14c2c86aeb5b60f838197, 5fc7387b9219407387e9f80b87e44a51; no voters:
I20260806 20:25:49.827337 26376 raft_consensus.cc:2781] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:49.827514 26376 raft_consensus.cc:686] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc7387b9219407387e9f80b87e44a51, State: Running, Role: LEADER
I20260806 20:25:49.827821 26376 consensus_queue.cc:237] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } }
I20260806 20:25:49.830686 25821 catalog_manager.cc:5495] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "5fc7387b9219407387e9f80b87e44a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fc7387b9219407387e9f80b87e44a51" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 46855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 } health_report { overall_health: UNKNOWN } } }
I20260806 20:25:49.855683 26294 raft_consensus.cc:1261] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:49.855715 25990 raft_consensus.cc:1261] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Refusing update from remote peer 5fc7387b9219407387e9f80b87e44a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:25:49.856283 26381 consensus_queue.cc:1035] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093d3938c8c14c2c86aeb5b60f838197" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:49.856873 26381 consensus_queue.cc:1035] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb4881f92bc43f9be696e27659442ea" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:25:49.899670 25821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:49.899935 25821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:25:49.902758 25821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56e216e598bc4b6f8f14fcc40ff54c78: Sending DeleteTablet for 3 replicas of tablet 9ae272bc58bc463cac53223143ced5e2
I20260806 20:25:49.903491 25969 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae272bc58bc463cac53223143ced5e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:54480
I20260806 20:25:49.903584 26115 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae272bc58bc463cac53223143ced5e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:56752
I20260806 20:25:49.903847 26433 tablet_replica.cc:330] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: stopping tablet replica
I20260806 20:25:49.903789 26269 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae272bc58bc463cac53223143ced5e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:25:49 UTC) from {username='oryx', principal='oryx/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:42036
I20260806 20:25:49.903985 26435 tablet_replica.cc:330] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: stopping tablet replica
I20260806 20:25:49.904049 26433 raft_consensus.cc:2229] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:49.904191 26435 raft_consensus.cc:2229] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:25:49.904282 26433 raft_consensus.cc:2258] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.904575 26434 tablet_replica.cc:330] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: stopping tablet replica
I20260806 20:25:49.904551 26435 raft_consensus.cc:2258] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.904673 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 25900
I20260806 20:25:49.904824 26434 raft_consensus.cc:2229] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:25:49.905027 26434 raft_consensus.cc:2258] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:25:49.906451 26435 ts_tablet_manager.cc:1905] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.906591 26434 ts_tablet_manager.cc:1905] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.916267 26435 ts_tablet_manager.cc:1918] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:49.916467 26435 log.cc:1196] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/9ae272bc58bc463cac53223143ced5e2
I20260806 20:25:49.916532 26433 ts_tablet_manager.cc:1905] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:25:49.917065 26435 ts_tablet_manager.cc:1939] T 9ae272bc58bc463cac53223143ced5e2 P 5fc7387b9219407387e9f80b87e44a51: Deleting consensus metadata
I20260806 20:25:49.918618 25808 catalog_manager.cc:4841] TS 5fc7387b9219407387e9f80b87e44a51 (127.17.109.194:46855): tablet 9ae272bc58bc463cac53223143ced5e2 (table test-table [id=b43a21392f094ff9b332235cb5541696]) successfully deleted
I20260806 20:25:49.919896 26434 ts_tablet_manager.cc:1918] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:49.920094 26434 log.cc:1196] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/9ae272bc58bc463cac53223143ced5e2
I20260806 20:25:49.920555 26434 ts_tablet_manager.cc:1939] T 9ae272bc58bc463cac53223143ced5e2 P 093d3938c8c14c2c86aeb5b60f838197: Deleting consensus metadata
I20260806 20:25:49.921547 25807 catalog_manager.cc:4841] TS 093d3938c8c14c2c86aeb5b60f838197 (127.17.109.195:33139): tablet 9ae272bc58bc463cac53223143ced5e2 (table test-table [id=b43a21392f094ff9b332235cb5541696]) successfully deleted
I20260806 20:25:49.923048 26433 ts_tablet_manager.cc:1918] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 20:25:49.923184 26433 log.cc:1196] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/9ae272bc58bc463cac53223143ced5e2
I20260806 20:25:49.923671 26433 ts_tablet_manager.cc:1939] T 9ae272bc58bc463cac53223143ced5e2 P 8eb4881f92bc43f9be696e27659442ea: Deleting consensus metadata
I20260806 20:25:49.924633 25809 catalog_manager.cc:4841] TS 8eb4881f92bc43f9be696e27659442ea (127.17.109.193:38741): tablet 9ae272bc58bc463cac53223143ced5e2 (table test-table [id=b43a21392f094ff9b332235cb5541696]) successfully deleted
Aug 06 20:25:49 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.193@KRBTEST.COM
I20260806 20:25:49.935415 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:49.906337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.17.109.193:38741)
0806 20:25:49.919472 (+ 13135us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:49.919477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:49.919495 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:25:49.919547 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:49.919550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:49.919680 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:49.919852 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:49.921671 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.922676 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.924535 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:49.925024 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:49.925122 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:49.929416 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:49.929433 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:49.929450 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:49.929484 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:49.932567 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:49.933286 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:49.933294 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:49.933296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:49.933422 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:49.933815 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:49.933819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:49.933821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:49.934194 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:49.934507 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:49.934809 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 20:25:49.935058 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13012,"thread_start_us":12889,"threads_started":1}
I20260806 20:25:49.936096 26039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:50.034226 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 25900
I20260806 20:25:50.055146 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 26049
Aug 06 20:25:50 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.194@KRBTEST.COM
I20260806 20:25:50.070340 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:50.056017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47014 (local address 127.17.109.194:46855)
0806 20:25:50.056204 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:50.056209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:50.056242 (+ 33us) server_negotiation.cc:407] Connection header received
0806 20:25:50.056428 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:50.056432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:50.056499 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:50.056588 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:50.057780 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:50.058501 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:50.059779 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:50.060184 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:50.060300 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:50.064393 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:50.064407 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:50.064430 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:50.064470 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:50.067350 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:50.068024 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:50.068032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:50.068034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:50.068165 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:50.068812 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:50.068817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:50.068819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:50.069269 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:50.069525 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:50.069835 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 20:25:50.070037 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:25:50.071151 26185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:50.222774 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 26049
I20260806 20:25:50.251806 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 26194
Aug 06 20:25:50 dist-test-slave-6k22 krb5kdc[25776](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.109.195@KRBTEST.COM
I20260806 20:25:50.267002 26470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:50.252551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52810 (local address 127.17.109.195:33139)
0806 20:25:50.253049 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:50.253053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:50.253070 (+ 17us) server_negotiation.cc:407] Connection header received
0806 20:25:50.253124 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:50.253127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:50.253200 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:50.253295 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:50.254572 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:50.255283 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:50.256586 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:50.256996 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:50.257094 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:50.261524 (+ 4430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:50.261538 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:50.261554 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:50.261587 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:50.264526 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:50.265176 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:50.265184 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:50.265186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:50.265330 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:50.265633 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:50.265636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:50.265638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:50.266035 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:50.266261 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:50.266525 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 20:25:50.266732 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":302,"threads_started":1}
I20260806 20:25:50.267740 26336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:50.377758 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 26194
I20260806 20:25:50.400883 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 25789
I20260806 20:25:50.402333 25868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:50.528100 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 25789
2026-08-06T20:25:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6010 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:50 dist-test-slave-6k22 krb5kdc[26475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:50 dist-test-slave-6k22 krb5kdc[26475](info): set up 2 sockets
Aug 06 20:25:50 dist-test-slave-6k22 krb5kdc[26475](info): commencing operation
krb5kdc: starting...
W20260806 20:25:51.728425 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.015s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:51 dist-test-slave-6k22 krb5kdc[26475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:25:51.996973 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:36951
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:36951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:52.329129 26488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:52.329442 26488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:52.329535 26488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:52.336719 26488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:52.336831 26488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:52.336879 26488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:52.336928 26488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:52.336977 26488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:52.346666 26488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:36951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:36951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:52.347954 26488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:52.349243 26488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:52.361245 26493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:52.361184 26496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:52.361085 26494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:52.362919 26488 server_base.cc:1034] running on GCE node
I20260806 20:25:52.363787 26488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:52.365279 26488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:52.366514 26488 hybrid_clock.cc:648] HybridClock initialized: now 1786047952366445 us; error 63 us; skew 500 ppm
I20260806 20:25:52.366870 26488 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T20:25:52Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1915 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:25:52 dist-test-slave-6k22 krb5kdc[26502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:25:52 dist-test-slave-6k22 krb5kdc[26502](info): set up 2 sockets
Aug 06 20:25:52 dist-test-slave-6k22 krb5kdc[26502](info): commencing operation
krb5kdc: starting...
W20260806 20:25:53.617791 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:25:53 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:25:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:25:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:25:53.901571 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38721
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:38721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:54.246932 26517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:54.247383 26517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:54.247575 26517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:54.258103 26517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:54.258258 26517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:54.258340 26517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:54.258464 26517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:54.258530 26517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:54.271854 26517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:38721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:54.273483 26517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:54.275414 26517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:54.290812 26522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:54.290859 26524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:54.292378 26526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:54.297394 26517 server_base.cc:1034] running on GCE node
I20260806 20:25:54.298183 26517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:54.299757 26517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:54.301002 26517 hybrid_clock.cc:648] HybridClock initialized: now 1786047954300958 us; error 38 us; skew 500 ppm
I20260806 20:25:54.301374 26517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:54 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047954, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:54.304715 26517 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:25:54.306444 26517 webserver.cc:460] Webserver started at http://127.17.109.254:34235/ using document root <none> and password file <none>
I20260806 20:25:54.307210 26517 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:54.307368 26517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:54.307787 26517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:54.310889 26517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "bdaccc0164dc4263bc95c6205fdba434"
format_stamp: "Formatted at 2026-08-06 20:25:54 on dist-test-slave-6k22"
server_key: "4925c3facc70b1638dbb81d7ea351db2"
server_key_iv: "22b0480842f67c77c67419ba57aafa50"
server_key_version: "encryptionkey@0"
I20260806 20:25:54.311766 26517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "bdaccc0164dc4263bc95c6205fdba434"
format_stamp: "Formatted at 2026-08-06 20:25:54 on dist-test-slave-6k22"
server_key: "4925c3facc70b1638dbb81d7ea351db2"
server_key_iv: "22b0480842f67c77c67419ba57aafa50"
server_key_version: "encryptionkey@0"
I20260806 20:25:54.317183 26517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 20:25:54.320887 26533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:54.322259 26517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:25:54.322464 26517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "bdaccc0164dc4263bc95c6205fdba434"
format_stamp: "Formatted at 2026-08-06 20:25:54 on dist-test-slave-6k22"
server_key: "4925c3facc70b1638dbb81d7ea351db2"
server_key_iv: "22b0480842f67c77c67419ba57aafa50"
server_key_version: "encryptionkey@0"
I20260806 20:25:54.322695 26517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:54.334684 26517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:54.338951 26517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:54.339255 26517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:54.359002 26517 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:38721
I20260806 20:25:54.359090 26591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:38721 every 8 connection(s)
I20260806 20:25:54.360579 26517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:54.364446 26592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:25:54.369714 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 26517
I20260806 20:25:54.369961 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:25:54.370725 17847 external_mini_cluster.cc:1428] Setting key 630fe9d0e65a9b49a791abfdc01f3798
I20260806 20:25:54.376205 26592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Bootstrap starting.
I20260806 20:25:54.380136 26592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Neither blocks nor log segments found. Creating new log.
I20260806 20:25:54.381410 26592 log.cc:824] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:54.385172 26592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: No bootstrap required, opened a new log
Aug 06 20:25:54 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:54.396298 26592 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } }
I20260806 20:25:54.396790 26592 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:25:54.396986 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:54.372858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54558 (local address 127.17.109.254:38721)
0806 20:25:54.373730 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:54.373780 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:54.373822 (+ 42us) server_negotiation.cc:407] Connection header received
0806 20:25:54.375545 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:54.375627 (+ 82us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:54.376019 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:54.376401 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:54.378235 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:54.379543 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:54.381455 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:54.382616 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:54.382868 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:54.387460 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:54.387470 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:54.387496 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:54.387548 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:54.392042 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:54.392829 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:54.392839 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:54.392848 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:54.393077 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:54.393443 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:54.393451 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:54.393456 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:54.394281 (+ 825us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:54.394607 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:54.394985 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 20:25:54.395473 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":418,"threads_started":1}
I20260806 20:25:54.396963 26592 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdaccc0164dc4263bc95c6205fdba434, State: Initialized, Role: FOLLOWER
I20260806 20:25:54.397598 26592 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } }
I20260806 20:25:54.397866 26592 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:54.398013 26592 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:54.398216 26592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:25:54.400663 26592 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } }
I20260806 20:25:54.401149 26592 leader_election.cc:302] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdaccc0164dc4263bc95c6205fdba434; no voters:
I20260806 20:25:54.401727 26592 leader_election.cc:288] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:25:54.401870 26597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:25:54.402598 26597 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 1 LEADER]: Becoming Leader. State: Replica: bdaccc0164dc4263bc95c6205fdba434, State: Running, Role: LEADER
I20260806 20:25:54.403165 26597 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } }
I20260806 20:25:54.403846 26592 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:54.406069 26598 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdaccc0164dc4263bc95c6205fdba434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } } }
I20260806 20:25:54.406836 26598 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:54.407346 26599 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdaccc0164dc4263bc95c6205fdba434. Latest consensus state: current_term: 1 leader_uuid: "bdaccc0164dc4263bc95c6205fdba434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } } }
I20260806 20:25:54.407788 26599 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:54.408752 26606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:54.416283 26606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:54.424921 26606 catalog_manager.cc:1347] Generated new cluster ID: 0b6e5bb86630421f8ac0f1f4268dfebf
I20260806 20:25:54.425042 26606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:54.434559 26606 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:25:54.436512 26606 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:54.445029 26606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Generated new TSK 0
I20260806 20:25:54.446027 26606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:25:54.514981 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38721
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:54.853502 26621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:54.853801 26621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:54.853899 26621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:54.861761 26621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:54.861882 26621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:54.861995 26621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:54.871533 26621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:54.872769 26621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:54.873994 26621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:54.887715 26627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:54.887728 26626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:54.887660 26629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:54.893242 26621 server_base.cc:1034] running on GCE node
I20260806 20:25:54.893901 26621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:54.894902 26621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:54.896140 26621 hybrid_clock.cc:648] HybridClock initialized: now 1786047954896097 us; error 38 us; skew 500 ppm
I20260806 20:25:54.896471 26621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:54 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047954, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:54.900089 26621 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:25:54.901728 26621 webserver.cc:460] Webserver started at http://127.17.109.193:45163/ using document root <none> and password file <none>
I20260806 20:25:54.902418 26621 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:54.902519 26621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:54.902779 26621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:54.905884 26621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "37a4720af15f4fd5b954043512505785"
format_stamp: "Formatted at 2026-08-06 20:25:54 on dist-test-slave-6k22"
server_key: "3421ea3efc573e66d2213d500379edbf"
server_key_iv: "00e302f522365c168e4abb5ffe10dc15"
server_key_version: "encryptionkey@0"
I20260806 20:25:54.906634 26621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "37a4720af15f4fd5b954043512505785"
format_stamp: "Formatted at 2026-08-06 20:25:54 on dist-test-slave-6k22"
server_key: "3421ea3efc573e66d2213d500379edbf"
server_key_iv: "00e302f522365c168e4abb5ffe10dc15"
server_key_version: "encryptionkey@0"
I20260806 20:25:54.911834 26621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 20:25:54.915212 26636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:54.916550 26621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:54.916720 26621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "37a4720af15f4fd5b954043512505785"
format_stamp: "Formatted at 2026-08-06 20:25:54 on dist-test-slave-6k22"
server_key: "3421ea3efc573e66d2213d500379edbf"
server_key_iv: "00e302f522365c168e4abb5ffe10dc15"
server_key_version: "encryptionkey@0"
I20260806 20:25:54.916909 26621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:54.930344 26621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:54.935218 26621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:54.935488 26621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:54.936533 26621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:54.937979 26621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:54.938068 26621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:54.938259 26621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:54.938347 26621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:54.974496 26621 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:39597
I20260806 20:25:54.974597 26752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:39597 every 8 connection(s)
I20260806 20:25:54.975904 26621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:54.982640 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 26621
I20260806 20:25:54.982903 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:25:54.983662 17847 external_mini_cluster.cc:1428] Setting key 1e0bc014d67d144cf80b177a2953c795
Aug 06 20:25:54 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047954, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:54.997850 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:54.978687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:51901 (local address 127.17.109.254:38721)
0806 20:25:54.979159 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:54.979163 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:54.980075 (+ 912us) server_negotiation.cc:407] Connection header received
0806 20:25:54.981942 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:54.981947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:54.982040 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:54.982161 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:54.983773 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:54.984677 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:54.986455 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:54.986882 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:54.987021 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:54.990909 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:54.990914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:54.990917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:54.990950 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:54.994109 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:54.995005 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:54.995012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:54.995018 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:54.995147 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:54.995631 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:54.995638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:54.995643 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:54.995987 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:54.996113 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:54.997058 (+ 945us) server_negotiation.cc:299] Negotiation successful
0806 20:25:54.997357 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":264,"threads_started":1}
I20260806 20:25:54.998448 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:54.979146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:51901)
0806 20:25:54.979908 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:54.979957 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:54.981642 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:54.982405 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:54.982412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:54.982965 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:54.983608 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:54.983628 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:54.984830 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:54.984836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:54.986303 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:54.986327 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:54.987121 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:54.987130 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:54.987379 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:54.988198 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:54.988221 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:54.990672 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:54.994274 (+ 3602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:54.994284 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:54.994304 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:54.994872 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:54.995258 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:54.995264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:54.995270 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:54.995489 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:54.996179 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:54.996189 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:54.996615 (+ 426us) client_negotiation.cc:769] Sending connection context
0806 20:25:54.996850 (+ 235us) client_negotiation.cc:240] Negotiation successful
0806 20:25:54.997244 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":339,"threads_started":1}
I20260806 20:25:55.000226 26753 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38721
I20260806 20:25:55.000573 26753 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:55.001359 26753 heartbeater.cc:511] Master 127.17.109.254:38721 requested a full tablet report, sending...
I20260806 20:25:55.003602 26556 ts_manager.cc:194] Registered new tserver with Master: 37a4720af15f4fd5b954043512505785 (127.17.109.193:39597)
I20260806 20:25:55.005405 26556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:51901
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:25:55.048003 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38721
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:55.383122 26761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:55.383457 26761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:55.383709 26761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:55.390990 26761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:55.391146 26761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:55.391340 26761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:55.401475 26761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:55.402827 26761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:55.404249 26761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:55.418074 26767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:55.418191 26766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:55.418174 26769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:55.418777 26761 server_base.cc:1034] running on GCE node
I20260806 20:25:55.419396 26761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:55.420449 26761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:55.421720 26761 hybrid_clock.cc:648] HybridClock initialized: now 1786047955421633 us; error 75 us; skew 500 ppm
I20260806 20:25:55.422089 26761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:55 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047955, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:55.425493 26761 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:25:55.427106 26761 webserver.cc:460] Webserver started at http://127.17.109.194:46749/ using document root <none> and password file <none>
I20260806 20:25:55.427951 26761 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:55.428097 26761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:55.428412 26761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:55.432025 26761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "743b710e958645159f3220f673fddc05"
format_stamp: "Formatted at 2026-08-06 20:25:55 on dist-test-slave-6k22"
server_key: "79b7e6c29011de8a942a31515da6ff7f"
server_key_iv: "813ea71bd956e030f299c0faa324cab2"
server_key_version: "encryptionkey@0"
I20260806 20:25:55.433151 26761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "743b710e958645159f3220f673fddc05"
format_stamp: "Formatted at 2026-08-06 20:25:55 on dist-test-slave-6k22"
server_key: "79b7e6c29011de8a942a31515da6ff7f"
server_key_iv: "813ea71bd956e030f299c0faa324cab2"
server_key_version: "encryptionkey@0"
I20260806 20:25:55.438798 26761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260806 20:25:55.442454 26776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:55.444082 26761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:55.444273 26761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "743b710e958645159f3220f673fddc05"
format_stamp: "Formatted at 2026-08-06 20:25:55 on dist-test-slave-6k22"
server_key: "79b7e6c29011de8a942a31515da6ff7f"
server_key_iv: "813ea71bd956e030f299c0faa324cab2"
server_key_version: "encryptionkey@0"
I20260806 20:25:55.444483 26761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:55.457513 26761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:55.461454 26761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:55.461752 26761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:55.462651 26761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:55.464031 26761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:55.464133 26761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:55.464286 26761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:55.464360 26761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:55.498466 26761 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:32779
I20260806 20:25:55.498560 26895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:32779 every 8 connection(s)
I20260806 20:25:55.500018 26761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:55.506608 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 26761
I20260806 20:25:55.506848 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:25:55.507359 17847 external_mini_cluster.cc:1428] Setting key 539dcce8ba3bf4a0be001b7b778cd555
Aug 06 20:25:55 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047955, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:55.521682 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:55.502788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:40649 (local address 127.17.109.254:38721)
0806 20:25:55.503273 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:55.503280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:55.504467 (+ 1187us) server_negotiation.cc:407] Connection header received
0806 20:25:55.506435 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:55.506443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:55.506540 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:55.506708 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:55.508604 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:55.509362 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:55.510909 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:55.511347 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:55.511488 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:55.515473 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:55.515478 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:55.515481 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:55.515512 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:55.518147 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:55.518923 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:55.518932 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:55.518935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:55.519064 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:55.519515 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:55.519518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:55.519521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:55.519851 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:55.520037 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:55.520904 (+ 867us) server_negotiation.cc:299] Negotiation successful
0806 20:25:55.521155 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":222,"threads_started":1}
I20260806 20:25:55.522475 26899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:55.503436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:40649)
0806 20:25:55.504265 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:55.504333 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:55.506141 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:55.507034 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:55.507043 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:55.507711 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:55.508409 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:55.508435 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:55.509525 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:55.509532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:55.510753 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:55.510764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:55.511605 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:55.511613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:55.511905 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:55.512716 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:55.512740 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:55.515245 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:55.518318 (+ 3073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:55.518326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:55.518342 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:55.518790 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:55.519178 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:55.519184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:55.519188 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:55.519384 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:55.520026 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:55.520038 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:55.520450 (+ 412us) client_negotiation.cc:769] Sending connection context
0806 20:25:55.520768 (+ 318us) client_negotiation.cc:240] Negotiation successful
0806 20:25:55.521269 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":287,"threads_started":1}
I20260806 20:25:55.524125 26896 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38721
I20260806 20:25:55.524466 26896 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:55.525226 26896 heartbeater.cc:511] Master 127.17.109.254:38721 requested a full tablet report, sending...
I20260806 20:25:55.526714 26556 ts_manager.cc:194] Registered new tserver with Master: 743b710e958645159f3220f673fddc05 (127.17.109.194:32779)
I20260806 20:25:55.527702 26556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:40649
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:25:55.571489 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38721
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:55.905869 26902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:55.906224 26902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:55.906409 26902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:55.914134 26902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:55.914299 26902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:55.914485 26902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:55.924806 26902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:55.926184 26902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:55.927757 26902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:55.942066 26907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:55.942142 26910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:55.942710 26902 server_base.cc:1034] running on GCE node
W20260806 20:25:55.942845 26908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:55.943502 26902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:55.944653 26902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:55.945911 26902 hybrid_clock.cc:648] HybridClock initialized: now 1786047955945842 us; error 53 us; skew 500 ppm
I20260806 20:25:55.946231 26902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:55 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047955, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:55.949630 26902 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:25:55.951354 26902 webserver.cc:460] Webserver started at http://127.17.109.195:39657/ using document root <none> and password file <none>
I20260806 20:25:55.952108 26902 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:55.952247 26902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:55.952590 26902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:25:55.956015 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "4cc4ad84d9d742a1b496fd31910a9e6b"
format_stamp: "Formatted at 2026-08-06 20:25:55 on dist-test-slave-6k22"
server_key: "2ccadf82e4b72c8c6d0179cdc61c063a"
server_key_iv: "fd9c8f9f92126b2b2057b4dee8f93118"
server_key_version: "encryptionkey@0"
I20260806 20:25:55.956835 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "4cc4ad84d9d742a1b496fd31910a9e6b"
format_stamp: "Formatted at 2026-08-06 20:25:55 on dist-test-slave-6k22"
server_key: "2ccadf82e4b72c8c6d0179cdc61c063a"
server_key_iv: "fd9c8f9f92126b2b2057b4dee8f93118"
server_key_version: "encryptionkey@0"
I20260806 20:25:55.962111 26902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 20:25:55.965622 26917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:55.967116 26902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 20:25:55.967322 26902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "4cc4ad84d9d742a1b496fd31910a9e6b"
format_stamp: "Formatted at 2026-08-06 20:25:55 on dist-test-slave-6k22"
server_key: "2ccadf82e4b72c8c6d0179cdc61c063a"
server_key_iv: "fd9c8f9f92126b2b2057b4dee8f93118"
server_key_version: "encryptionkey@0"
I20260806 20:25:55.967574 26902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:55.982296 26902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:55.986575 26902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:55.986887 26902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:55.987951 26902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:55.989399 26902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:55.989511 26902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:55.989658 26902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:55.989738 26902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:56.008832 26753 heartbeater.cc:503] Master 127.17.109.254:38721 was elected leader, sending a full tablet report...
I20260806 20:25:56.025183 26902 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:33713
I20260806 20:25:56.025223 27038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:33713 every 8 connection(s)
I20260806 20:25:56.026520 26902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:56.029593 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 26902
I20260806 20:25:56.029835 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:25:56.030352 17847 external_mini_cluster.cc:1428] Setting key 06e0f5a8ce9d06a6472b53e7ec362c10
Aug 06 20:25:56 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047955, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:25:56.047787 27041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:56.029245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:35673 (local address 127.17.109.254:38721)
0806 20:25:56.029822 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:56.029828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:56.030945 (+ 1117us) server_negotiation.cc:407] Connection header received
0806 20:25:56.033040 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:56.033045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:56.033137 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:56.033249 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:56.034761 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.035611 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.037141 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.037552 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.037685 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:56.041158 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:56.041163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:56.041166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:56.041200 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:56.044417 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:56.045228 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:56.045234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:56.045242 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:56.045375 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:56.045832 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:56.045840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:56.045843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:56.046129 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:56.046302 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:56.047075 (+ 773us) server_negotiation.cc:299] Negotiation successful
0806 20:25:56.047313 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":273,"threads_started":1}
I20260806 20:25:56.048967 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:56.029806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:35673)
0806 20:25:56.030738 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:56.030793 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:56.032777 (+ 1984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:56.033497 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:56.033505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:56.034001 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:56.034560 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:56.034577 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.035772 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.035776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:56.036988 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:56.037000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.037775 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.037782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:56.038031 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:56.038694 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:56.038718 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:56.040948 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:25:56.044575 (+ 3627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:56.044585 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:56.044609 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:56.045077 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:56.045496 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:25:56.045501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:25:56.045505 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:25:56.045712 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:25:56.046265 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:25:56.046273 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:25:56.046647 (+ 374us) client_negotiation.cc:769] Sending connection context
0806 20:25:56.046983 (+ 336us) client_negotiation.cc:240] Negotiation successful
0806 20:25:56.047605 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":330,"threads_started":1}
I20260806 20:25:56.050513 27039 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38721
I20260806 20:25:56.050838 27039 heartbeater.cc:464] Registering TS with master...
I20260806 20:25:56.051671 27039 heartbeater.cc:511] Master 127.17.109.254:38721 requested a full tablet report, sending...
I20260806 20:25:56.053239 26556 ts_manager.cc:194] Registered new tserver with Master: 4cc4ad84d9d742a1b496fd31910a9e6b (127.17.109.195:33713)
I20260806 20:25:56.054160 26556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:35673
I20260806 20:25:56.058252 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab.
Entry for principal oryx/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab.
WARNING: no policy specified for oryx/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab.
Entry for principal oryx/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab.
WARNING: no policy specified for oryx/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab.
Entry for principal oryx/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab.
I20260806 20:25:56.227283 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 26621
Aug 06 20:25:56 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:25:56.243937 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:56.228569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.17.109.193:39597)
0806 20:25:56.229241 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:56.229248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:56.229277 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:25:56.229339 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:56.229346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:56.229560 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:56.229711 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:56.231128 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.232260 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.233556 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.234128 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.234230 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:56.238341 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:56.238352 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:56.238379 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:56.238414 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:56.241242 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:56.242016 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:56.242022 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:56.242025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:56.242146 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:56.242509 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:56.242512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:56.242514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:56.242845 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:56.243079 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:56.243361 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 20:25:56.243611 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":358,"threads_started":1}
I20260806 20:25:56.245322 26748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:56.329252 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 26621
I20260806 20:25:56.348773 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 26761
Aug 06 20:25:56 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:25:56.366255 27052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:56.349672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.17.109.194:32779)
0806 20:25:56.350313 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:56.350320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:56.350346 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:25:56.350406 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:56.350412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:56.350621 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:56.350752 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:56.352156 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.353306 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.354570 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.355106 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.355201 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:56.359524 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:56.359537 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:56.359586 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:56.359634 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:56.363257 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:56.364020 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:56.364031 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:56.364042 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:56.364205 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:56.364548 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:56.364552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:56.364556 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:56.364988 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:56.365281 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:56.365570 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 20:25:56.365850 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":340,"threads_started":1}
I20260806 20:25:56.367676 26891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:56.453410 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 26761
I20260806 20:25:56.472231 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 26902
Aug 06 20:25:56 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:25:56.489105 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:56.473217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34720 (local address 127.17.109.195:33713)
0806 20:25:56.473789 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:56.473796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:56.473827 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:25:56.473892 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:56.473899 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:56.474119 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:56.474275 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:56.475675 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.476646 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.478083 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:56.478658 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:56.478778 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:56.483156 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:56.483169 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:56.483185 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:56.483224 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:56.486077 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:56.486799 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:56.486807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:56.486810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:56.486944 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:56.487341 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:56.487348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:56.487351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:56.487783 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:56.488111 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:56.488438 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 20:25:56.488733 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":259,"threads_started":1}
I20260806 20:25:56.490589 27034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:56.575632 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 26902
I20260806 20:25:56.594545 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 26517
I20260806 20:25:56.595983 26587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:25:56.701610 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 26517
I20260806 20:25:56.725384 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38721
--webserver_interface=127.17.109.254
--webserver_port=34235
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:38721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:25:57.067335 27057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:57.067694 27057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:57.067854 27057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:57.075282 27057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:25:57.075388 27057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:57.075498 27057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:57.075618 27057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:25:57.075695 27057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:25:57.085808 27057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:38721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=34235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:57.087229 27057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:57.088622 27057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:57.101398 27062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:57.101363 27066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:57.101459 27063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:57.102567 27057 server_base.cc:1034] running on GCE node
I20260806 20:25:57.103443 27057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:57.105060 27057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:57.106324 27057 hybrid_clock.cc:648] HybridClock initialized: now 1786047957106276 us; error 78 us; skew 500 ppm
I20260806 20:25:57.106719 27057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:57 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047957, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:57.110208 27057 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:25:57.111894 27057 webserver.cc:460] Webserver started at http://127.17.109.254:34235/ using document root <none> and password file <none>
I20260806 20:25:57.112660 27057 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:57.112799 27057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:57.118959 27057 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 20:25:57.122052 27073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:57.123703 27057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260806 20:25:57.123919 27057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "bdaccc0164dc4263bc95c6205fdba434"
format_stamp: "Formatted at 2026-08-06 20:25:54 on dist-test-slave-6k22"
server_key: "4925c3facc70b1638dbb81d7ea351db2"
server_key_iv: "22b0480842f67c77c67419ba57aafa50"
server_key_version: "encryptionkey@0"
I20260806 20:25:57.125008 27057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:57.161285 27057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:57.165853 27057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:57.166155 27057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:57.186245 27057 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:38721
I20260806 20:25:57.186265 27126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:38721 every 8 connection(s)
I20260806 20:25:57.187911 27057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:25:57.193899 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 27057
I20260806 20:25:57.194682 27127 sys_catalog.cc:263] Verifying existing consensus state
I20260806 20:25:57.194842 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:39597
--local_ip_for_outbound_sockets=127.17.109.193
--tserver_master_addrs=127.17.109.254:38721
--webserver_port=45163
--webserver_interface=127.17.109.193
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.109.193
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:25:57.198071 27127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Bootstrap starting.
I20260806 20:25:57.220746 27127 log.cc:824] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Log is configured to *not* fsync() on all Append() calls
I20260806 20:25:57.233250 27127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:25:57.233907 27127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Bootstrap complete.
I20260806 20:25:57.247622 27127 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } }
I20260806 20:25:57.248788 27127 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdaccc0164dc4263bc95c6205fdba434, State: Initialized, Role: FOLLOWER
I20260806 20:25:57.249511 27127 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } }
I20260806 20:25:57.249819 27127 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:25:57.249953 27127 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:25:57.250133 27127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 1 FOLLOWER]: Advancing to term 2
I20260806 20:25:57.255667 27127 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } }
I20260806 20:25:57.256244 27127 leader_election.cc:302] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdaccc0164dc4263bc95c6205fdba434; no voters:
I20260806 20:25:57.256930 27127 leader_election.cc:288] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 20:25:57.257077 27131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 20:25:57.258666 27131 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [term 2 LEADER]: Becoming Leader. State: Replica: bdaccc0164dc4263bc95c6205fdba434, State: Running, Role: LEADER
I20260806 20:25:57.259264 27127 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:25:57.259231 27131 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } }
I20260806 20:25:57.261802 27133 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdaccc0164dc4263bc95c6205fdba434. Latest consensus state: current_term: 2 leader_uuid: "bdaccc0164dc4263bc95c6205fdba434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } } }
I20260806 20:25:57.262194 27133 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:57.262719 27132 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bdaccc0164dc4263bc95c6205fdba434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdaccc0164dc4263bc95c6205fdba434" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38721 } } }
I20260806 20:25:57.263018 27132 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434 [sys.catalog]: This master's current role is: LEADER
I20260806 20:25:57.273499 27140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:25:57.279389 27140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:25:57.281004 27140 catalog_manager.cc:1259] Loaded cluster ID: 0b6e5bb86630421f8ac0f1f4268dfebf
I20260806 20:25:57.281169 27140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:25:57.285292 27140 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:25:57.286876 27140 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bdaccc0164dc4263bc95c6205fdba434: Loaded TSK: 0
I20260806 20:25:57.288568 27140 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 20:25:57.565956 27129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:57.566334 27129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:57.566485 27129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:57.573971 27129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:57.574111 27129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:57.574295 27129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:25:57.584337 27129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.193.keytab
--principal=oryx/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:39597
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=45163
--tserver_master_addrs=127.17.109.254:38721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:57.585757 27129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:57.587237 27129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:57.601661 27155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:57.601723 27154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:57.601800 27157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:57.602535 27129 server_base.cc:1034] running on GCE node
I20260806 20:25:57.603369 27129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:57.604409 27129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:57.605672 27129 hybrid_clock.cc:648] HybridClock initialized: now 1786047957605590 us; error 55 us; skew 500 ppm
I20260806 20:25:57.606043 27129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:57 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047957, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:57.615648 27129 init.cc:377] Logged in from keytab as oryx/127.17.109.193@KRBTEST.COM (short username oryx)
I20260806 20:25:57.617565 27129 webserver.cc:460] Webserver started at http://127.17.109.193:45163/ using document root <none> and password file <none>
I20260806 20:25:57.618283 27129 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:57.618459 27129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:57.624529 27129 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 20:25:57.627384 27166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:57.628929 27129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:25:57.629174 27129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "37a4720af15f4fd5b954043512505785"
format_stamp: "Formatted at 2026-08-06 20:25:54 on dist-test-slave-6k22"
server_key: "3421ea3efc573e66d2213d500379edbf"
server_key_iv: "00e302f522365c168e4abb5ffe10dc15"
server_key_version: "encryptionkey@0"
I20260806 20:25:57.630220 27129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:57.647382 27129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:57.652205 27129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:57.652532 27129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:57.653657 27129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:57.655145 27129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:57.655282 27129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:57.655433 27129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:57.655536 27129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:57.692508 27129 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:39597
I20260806 20:25:57.692600 27283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:39597 every 8 connection(s)
I20260806 20:25:57.694024 27129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:25:57.699082 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 27129
I20260806 20:25:57.699775 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:32779
--local_ip_for_outbound_sockets=127.17.109.194
--tserver_master_addrs=127.17.109.254:38721
--webserver_port=46749
--webserver_interface=127.17.109.194
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.109.194
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:25:57 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:57 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:57.713336 27287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:57.697304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:44449)
0806 20:25:57.698120 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:57.698189 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:57.700077 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:57.701735 (+ 1658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:57.701742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:57.702251 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:57.702796 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:57.702814 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.705123 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.705127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:57.706335 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:57.706347 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.707832 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.707837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:57.708036 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:57.708750 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:57.708773 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:57.712364 (+ 3591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":345,"threads_started":1}
W20260806 20:25:57.714092 27284 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:38721 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.109.254:38721: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
W20260806 20:25:57.714547 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:57.697206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:44449 (local address 127.17.109.254:38721)
0806 20:25:57.697915 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:57.697925 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:57.698325 (+ 400us) server_negotiation.cc:407] Connection header received
0806 20:25:57.700697 (+ 2372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:57.700719 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:57.701229 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:57.701626 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:57.702997 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.704955 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.706486 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.707619 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.707820 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:57.712800 (+ 4980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:44449: BlockingRecv error: recv got EOF from 127.17.109.193:44449 (error 108)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":289,"threads_started":1}
Aug 06 20:25:57 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:57 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:57.725435 27287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:57.714793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:45067)
0806 20:25:57.715106 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:57.715127 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:57.715228 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:57.715694 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:57.715701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:57.716165 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:57.716401 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:57.716417 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.717525 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.717533 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:57.719222 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:57.719241 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.720078 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.720086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:57.720300 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:57.720946 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:57.720973 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:57.725068 (+ 4095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169}
W20260806 20:25:57.725455 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:57.715181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:45067 (local address 127.17.109.254:38721)
0806 20:25:57.715363 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:57.715368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:57.715386 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:25:57.715459 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:57.715463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:57.715520 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:57.715683 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:57.716568 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.717368 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.719394 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.719910 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.720034 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:57.725290 (+ 5256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:45067: BlockingRecv error: recv got EOF from 127.17.109.193:45067 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 20:25:57 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:57 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:57.735152 27287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:57.726280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:39699)
0806 20:25:57.726458 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:57.726480 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:57.726583 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:57.727010 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:57.727013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:57.727290 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:57.727429 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:57.727436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.728988 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.728991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:57.730079 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:57.730090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.731202 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.731207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:57.731356 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:57.732053 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:57.732070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:57.734885 (+ 2815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 20:25:57.735445 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:57.726416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:39699 (local address 127.17.109.254:38721)
0806 20:25:57.726602 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:57.726656 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:57.726680 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:25:57.726746 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:57.726754 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:57.726854 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:57.726964 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:57.727598 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.728842 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.730220 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:57.731030 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:57.731184 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:57.735128 (+ 3944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:39699: BlockingRecv error: recv got EOF from 127.17.109.193:39699 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 20:25:57.735646 27284 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 20:25:58.042627 27288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:58.042948 27288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:58.043149 27288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:58.050891 27288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:58.051019 27288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:58.051251 27288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:25:58.063108 27288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.194.keytab
--principal=oryx/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:32779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=46749
--tserver_master_addrs=127.17.109.254:38721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:58.064507 27288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:58.065860 27288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:58.079465 27297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:58.079569 27294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:58.079530 27295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:58.080610 27288 server_base.cc:1034] running on GCE node
I20260806 20:25:58.081355 27288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:58.082334 27288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:58.083666 27288 hybrid_clock.cc:648] HybridClock initialized: now 1786047958083605 us; error 33 us; skew 500 ppm
I20260806 20:25:58.084009 27288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:58.087347 27288 init.cc:377] Logged in from keytab as oryx/127.17.109.194@KRBTEST.COM (short username oryx)
I20260806 20:25:58.088985 27288 webserver.cc:460] Webserver started at http://127.17.109.194:46749/ using document root <none> and password file <none>
I20260806 20:25:58.089743 27288 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:58.089913 27288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:58.095396 27288 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:25:58.098213 27304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:58.099726 27288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:58.099900 27288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "743b710e958645159f3220f673fddc05"
format_stamp: "Formatted at 2026-08-06 20:25:55 on dist-test-slave-6k22"
server_key: "79b7e6c29011de8a942a31515da6ff7f"
server_key_iv: "813ea71bd956e030f299c0faa324cab2"
server_key_version: "encryptionkey@0"
I20260806 20:25:58.100883 27288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:58.120973 27288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:58.125197 27288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:58.125483 27288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:58.126459 27288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:58.127820 27288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:58.127945 27288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:58.128069 27288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:58.128155 27288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:58.162989 27288 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:32779
I20260806 20:25:58.163077 27422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:32779 every 8 connection(s)
I20260806 20:25:58.164477 27288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:25:58.172611 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 27288
I20260806 20:25:58.173380 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:33713
--local_ip_for_outbound_sockets=127.17.109.195
--tserver_master_addrs=127.17.109.254:38721
--webserver_port=39657
--webserver_interface=127.17.109.195
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.109.195
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 20:25:58.184108 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:58.167008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:35185 (local address 127.17.109.254:38721)
0806 20:25:58.167220 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:58.167227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:58.168444 (+ 1217us) server_negotiation.cc:407] Connection header received
0806 20:25:58.170154 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:58.170158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:58.170227 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:58.170370 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:58.171953 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.173073 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.175995 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.176760 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.176908 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.183788 (+ 6880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:35185: BlockingRecv error: recv got EOF from 127.17.109.194:35185 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 20:25:58.184633 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:58.167535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:35185)
0806 20:25:58.168286 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:58.168323 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:58.169903 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:58.170609 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:58.170617 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:58.171113 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:58.171768 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.171788 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.173303 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.173311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.175815 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.175834 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.177102 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.177113 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.177510 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.178381 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:58.178423 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:58.183436 (+ 5013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":257,"threads_started":1}
W20260806 20:25:58.185321 27423 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:38721 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.109.254:38721: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:58.194164 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:58.186030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:52257)
0806 20:25:58.186316 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:58.186336 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:58.186484 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:58.186833 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:58.186837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:58.187198 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:58.187348 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.187356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.188469 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.188474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.189411 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.189423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.190150 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.190156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.190337 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.190959 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:58.190976 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:58.193874 (+ 2898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136}
W20260806 20:25:58.194368 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:58.186108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:52257 (local address 127.17.109.254:38721)
0806 20:25:58.186302 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:58.186308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:58.186415 (+ 107us) server_negotiation.cc:407] Connection header received
0806 20:25:58.186616 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:58.186621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:58.186697 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:58.186807 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:58.187492 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.188326 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.189551 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.189991 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.190138 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.194104 (+ 3966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:52257: BlockingRecv error: recv got EOF from 127.17.109.194:52257 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:58.203598 27425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:58.195227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:60065)
0806 20:25:58.195451 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:58.195472 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:58.195605 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:58.196170 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:58.196175 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:58.196476 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:58.196613 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.196621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.197632 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.197637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.198550 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.198561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.199418 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.199423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.199669 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.200358 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:58.200386 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:58.203173 (+ 2787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132}
W20260806 20:25:58.203621 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:58.195301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:60065 (local address 127.17.109.254:38721)
0806 20:25:58.195915 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:58.195919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:58.195933 (+ 14us) server_negotiation.cc:407] Connection header received
0806 20:25:58.195971 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:58.195974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:58.196029 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:58.196138 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:58.196758 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.197496 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.198688 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.199247 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.199395 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.203381 (+ 3986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:60065: BlockingRecv error: recv got EOF from 127.17.109.194:60065 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 20:25:58.204203 27423 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 20:25:58.529089 27426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:25:58.529376 27426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:25:58.529471 27426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:25:58.537184 27426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:25:58.537305 27426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:25:58.537421 27426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:25:58.547418 27426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/krb5kdc/oryx_127.17.109.195.keytab
--principal=oryx/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:33713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=39657
--tserver_master_addrs=127.17.109.254:38721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:25:58.548815 27426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:25:58.550100 27426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:25:58.563987 27432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:58.563959 27431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:25:58.564124 27434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:25:58.564868 27426 server_base.cc:1034] running on GCE node
I20260806 20:25:58.565523 27426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:25:58.566521 27426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:25:58.567787 27426 hybrid_clock.cc:648] HybridClock initialized: now 1786047958567736 us; error 42 us; skew 500 ppm
I20260806 20:25:58.568131 27426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:25:58.571511 27426 init.cc:377] Logged in from keytab as oryx/127.17.109.195@KRBTEST.COM (short username oryx)
I20260806 20:25:58.573302 27426 webserver.cc:460] Webserver started at http://127.17.109.195:39657/ using document root <none> and password file <none>
I20260806 20:25:58.574064 27426 fs_manager.cc:359] Metadata directory not provided
I20260806 20:25:58.574240 27426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:25:58.580334 27426 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 20:25:58.583458 27441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:58.585207 27426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:25:58.585430 27426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "4cc4ad84d9d742a1b496fd31910a9e6b"
format_stamp: "Formatted at 2026-08-06 20:25:55 on dist-test-slave-6k22"
server_key: "2ccadf82e4b72c8c6d0179cdc61c063a"
server_key_iv: "fd9c8f9f92126b2b2057b4dee8f93118"
server_key_version: "encryptionkey@0"
I20260806 20:25:58.586570 27426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:25:58.600186 27426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:25:58.604779 27426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:25:58.605072 27426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:25:58.606104 27426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:25:58.607431 27426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:25:58.607525 27426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:58.607694 27426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:25:58.607795 27426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:25:58.640389 27426 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:33713
I20260806 20:25:58.640444 27561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:33713 every 8 connection(s)
I20260806 20:25:58.641894 27426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:25:58.643704 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 27426
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 20:25:58.663388 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:58.644627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:33049 (local address 127.17.109.254:38721)
0806 20:25:58.644913 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:58.644922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:58.648560 (+ 3638us) server_negotiation.cc:407] Connection header received
0806 20:25:58.652276 (+ 3716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:58.652283 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:58.652399 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:58.652522 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:58.654413 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.655245 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.656930 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.657578 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.657719 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.663093 (+ 5374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:33049: BlockingRecv error: recv got EOF from 127.17.109.195:33049 (error 108)
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":93}
I20260806 20:25:58.664009 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:58.645297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:33049)
0806 20:25:58.648325 (+ 3028us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:58.648402 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:58.651868 (+ 3466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:58.652863 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:58.652875 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:58.653537 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:58.654225 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.654244 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.655412 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.655418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.656760 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.656778 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.657834 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.657839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.658041 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.658737 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:58.658760 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:58.662841 (+ 4081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2624,"thread_start_us":2353,"threads_started":1}
W20260806 20:25:58.664685 27562 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:38721 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.109.254:38721: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
I20260806 20:25:58.667869 27565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:58.644814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.17.109.254:38721)
0806 20:25:58.651989 (+ 7175us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:58.651993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:58.652599 (+ 606us) server_negotiation.cc:407] Connection header received
0806 20:25:58.652770 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:58.652777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:58.652850 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:58.652962 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:58.654548 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.655448 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.656928 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.657420 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.657514 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.660659 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:25:58.660670 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:25:58.660694 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:25:58.660730 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:25:58.664132 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:58.664985 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:58.664999 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:58.665007 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:58.665221 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:25:58.665789 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:25:58.665794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:25:58.665798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:25:58.666337 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:25:58.666707 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0806 20:25:58.667130 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 20:25:58.667573 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":7083,"thread_start_us":6951,"threads_started":1}
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:58.673220 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:58.665307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:51855)
0806 20:25:58.665498 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:58.665518 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:58.665613 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:58.665908 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:58.665912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:58.666254 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:58.666386 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.666394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.667381 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.667386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.668509 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.668521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.669443 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.669447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.669565 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.670140 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:58.670162 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:58.673035 (+ 2873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 20:25:58.673552 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:58.665407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:51855 (local address 127.17.109.254:38721)
0806 20:25:58.665583 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:58.665588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:58.665604 (+ 16us) server_negotiation.cc:407] Connection header received
0806 20:25:58.665718 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:58.665723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:58.665785 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:58.665882 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:58.666527 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.667246 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.668646 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.669301 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.669427 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.673300 (+ 3873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:51855: BlockingRecv error: recv got EOF from 127.17.109.195:51855 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:58.681586 27566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:58.673983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:47029)
0806 20:25:58.674161 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:58.674180 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:58.674284 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:58.674660 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:58.674665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:58.675006 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:58.675149 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.675157 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.676217 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.676225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.677148 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.677159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.677845 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.677851 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.678023 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.678611 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:58.678628 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:58.681259 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 20:25:58.681707 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:58.674094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:47029 (local address 127.17.109.254:38721)
0806 20:25:58.674321 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:58.674329 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:58.674350 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:25:58.674399 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:58.674404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:58.674487 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:58.674605 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:58.675294 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.676081 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.677298 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.677696 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.677822 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.681452 (+ 3630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:47029: BlockingRecv error: recv got EOF from 127.17.109.195:47029 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
W20260806 20:25:58.682122 27562 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:58 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:58.745240 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:58.736096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:34247)
0806 20:25:58.736730 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:58.736747 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:58.736861 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:58.737171 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:58.737174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:58.737649 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:58.737890 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.737899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.738915 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.738918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.739910 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:58.739921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.740711 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.740717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:58.740838 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.741491 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:58.741508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:58.744956 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":369,"threads_started":1}
W20260806 20:25:58.745456 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:58.736180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:34247 (local address 127.17.109.254:38721)
0806 20:25:58.736375 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:58.736379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:58.736826 (+ 447us) server_negotiation.cc:407] Connection header received
0806 20:25:58.736959 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:58.736962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:58.737027 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:58.737117 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:58.738034 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.738786 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.740068 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:58.740508 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:58.740609 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:58.745197 (+ 4588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:34247: BlockingRecv error: recv got EOF from 127.17.109.193:34247 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 20:25:59 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:59 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:59.213388 27568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:59.204862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:53553)
0806 20:25:59.205331 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:59.205347 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:59.205464 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:59.205759 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:59.205762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:59.206238 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:59.206434 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:59.206444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.207457 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.207461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:59.208628 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:59.208641 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.209389 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.209396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:59.209558 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:59.210168 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:59.210185 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:59.213043 (+ 2858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":251,"threads_started":1}
W20260806 20:25:59.213516 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:59.204870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:53553 (local address 127.17.109.254:38721)
0806 20:25:59.205133 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:59.205141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:59.205434 (+ 293us) server_negotiation.cc:407] Connection header received
0806 20:25:59.205562 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:59.205566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:59.205628 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:59.205718 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:59.206568 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.207329 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.208773 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.209202 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.209352 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:59.213246 (+ 3894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:53553: BlockingRecv error: recv got EOF from 127.17.109.194:53553 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 20:25:59 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:59 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:59.692235 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:59.682826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:57521)
0806 20:25:59.683432 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:59.683470 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:59.683725 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:59.684186 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:59.684190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:59.684878 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:59.685144 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:59.685154 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.686442 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.686446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:59.687435 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:59.687446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.688233 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.688237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:59.688353 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:59.688974 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:59.688992 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:59.691953 (+ 2961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":332,"threads_started":1}
W20260806 20:25:59.692451 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:59.682894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:57521 (local address 127.17.109.254:38721)
0806 20:25:59.683241 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:59.683247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:59.683590 (+ 343us) server_negotiation.cc:407] Connection header received
0806 20:25:59.683930 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:59.683935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:59.684011 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:59.684157 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:59.685334 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.686288 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.687607 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.688073 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.688195 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:59.692207 (+ 4012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:57521: BlockingRecv error: recv got EOF from 127.17.109.195:57521 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 06 20:25:59 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:25:59 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:25:59.755635 27570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:25:59.746342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:59475)
0806 20:25:59.746939 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 20:25:59.746956 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:25:59.747136 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:25:59.747546 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:25:59.747551 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:25:59.748052 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:25:59.748238 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:59.748248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.749371 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.749380 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:59.750392 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:25:59.750403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.751202 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.751209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:25:59.751403 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:59.752113 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:25:59.752131 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:25:59.755238 (+ 3107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":358,"threads_started":1}
W20260806 20:25:59.755780 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:25:59.746415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:59475 (local address 127.17.109.254:38721)
0806 20:25:59.746630 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 20:25:59.746640 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:25:59.747056 (+ 416us) server_negotiation.cc:407] Connection header received
0806 20:25:59.747286 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:25:59.747291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:25:59.747368 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:25:59.747495 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:25:59.748399 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.749219 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.750546 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:25:59.751015 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:25:59.751174 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:25:59.755493 (+ 4319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:59475: BlockingRecv error: recv got EOF from 127.17.109.193:59475 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 20:26:00 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:00 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:00.223162 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:00.214413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:37931)
0806 20:26:00.214920 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:00.214938 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:00.215059 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:00.215347 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:00.215349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:00.215801 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:00.215976 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:00.215986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.216945 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.216949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:00.217850 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:00.217861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.218559 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.218563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:00.218671 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:00.219372 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:00.219389 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:00.222787 (+ 3398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":275,"threads_started":1}
W20260806 20:26:00.223308 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:00.214446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:37931 (local address 127.17.109.254:38721)
0806 20:26:00.214673 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:00.214683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:00.215021 (+ 338us) server_negotiation.cc:407] Connection header received
0806 20:26:00.215161 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:00.215165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:00.215226 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:00.215308 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:00.216103 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.216824 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.217999 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.218426 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.218520 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:00.223060 (+ 4540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:37931: BlockingRecv error: recv got EOF from 127.17.109.194:37931 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 06 20:26:00 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:00 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:00.702870 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:00.693328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:50191)
0806 20:26:00.693893 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:00.693915 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:00.694067 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:00.694460 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:00.694467 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:00.695026 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:00.695309 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:00.695323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.696497 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.696504 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:00.697920 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:00.697933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.698652 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.698656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:00.698806 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:00.699429 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:00.699448 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:00.702506 (+ 3058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":299,"threads_started":1}
W20260806 20:26:00.703024 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:00.693587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:50191 (local address 127.17.109.254:38721)
0806 20:26:00.693778 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:00.693784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:00.694023 (+ 239us) server_negotiation.cc:407] Connection header received
0806 20:26:00.694200 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:00.694204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:00.694295 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:00.694378 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:00.695461 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.696336 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.698050 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.698492 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.698586 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:00.702735 (+ 4149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:50191: BlockingRecv error: recv got EOF from 127.17.109.195:50191 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 20:26:00 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:00 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:00.766670 27573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:00.756939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:36585)
0806 20:26:00.757976 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:00.757994 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:00.758211 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:00.758606 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:00.758610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:00.758969 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:00.759136 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:00.759145 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.760350 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.760355 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:00.761542 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:00.761557 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.762288 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.762296 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:00.762449 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:00.763241 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:00.763261 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:00.766387 (+ 3126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":736,"threads_started":1}
W20260806 20:26:00.766830 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:00.757022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:36585 (local address 127.17.109.254:38721)
0806 20:26:00.757301 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:00.757309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:00.758099 (+ 790us) server_negotiation.cc:407] Connection header received
0806 20:26:00.758356 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:00.758360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:00.758435 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:00.758517 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:00.759300 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.760202 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.761710 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:00.762128 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:00.762241 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:00.766581 (+ 4340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:36585: BlockingRecv error: recv got EOF from 127.17.109.193:36585 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 06 20:26:01 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:01 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:01.233999 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:01.224234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:50037)
0806 20:26:01.224894 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:01.224926 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:01.225062 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:01.225441 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:01.225446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:01.225869 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:01.226073 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:01.226084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.227210 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.227215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:01.228293 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:01.228309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.229074 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.229080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:01.229281 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:01.230090 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:01.230112 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:01.233727 (+ 3615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":405,"threads_started":1}
W20260806 20:26:01.234232 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:01.224309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:50037 (local address 127.17.109.254:38721)
0806 20:26:01.224485 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:01.224490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:01.225008 (+ 518us) server_negotiation.cc:407] Connection header received
0806 20:26:01.225209 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:01.225213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:01.225279 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:01.225388 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:01.226250 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.227039 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.228483 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.228923 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.229026 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:01.233935 (+ 4909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:50037: BlockingRecv error: recv got EOF from 127.17.109.194:50037 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 20:26:01 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:01 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:01.713127 27575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:01.704217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:59607)
0806 20:26:01.704801 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:01.704839 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:01.705011 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:01.705571 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:01.705576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:01.706004 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:01.706244 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:01.706255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.707343 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.707350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:01.708335 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:01.708346 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.709087 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.709093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:01.709285 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:01.709867 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:01.709885 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:01.712855 (+ 2970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":291,"threads_started":1}
W20260806 20:26:01.713342 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:01.704284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:59607 (local address 127.17.109.254:38721)
0806 20:26:01.704543 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:01.704551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:01.704995 (+ 444us) server_negotiation.cc:407] Connection header received
0806 20:26:01.705177 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:01.705183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:01.705296 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:01.705414 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:01.706392 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.707167 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.708484 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.708914 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.709071 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:01.713081 (+ 4010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:59607: BlockingRecv error: recv got EOF from 127.17.109.195:59607 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 06 20:26:01 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:01 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:01.775906 27576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:01.767682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:44101)
0806 20:26:01.768222 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:01.768238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:01.768355 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:01.768655 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:01.768658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:01.769045 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:01.769192 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:01.769201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.770134 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.770138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:01.771154 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:01.771166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.771909 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.771916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:01.772063 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:01.772627 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:01.772643 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:01.775668 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":307,"threads_started":1}
W20260806 20:26:01.776095 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:01.767790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:44101 (local address 127.17.109.254:38721)
0806 20:26:01.767973 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:01.767977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:01.768315 (+ 338us) server_negotiation.cc:407] Connection header received
0806 20:26:01.768453 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:01.768457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:01.768518 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:01.768597 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:01.769313 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.770018 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.771289 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:01.771754 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:01.771848 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:01.775869 (+ 4021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:44101: BlockingRecv error: recv got EOF from 127.17.109.193:44101 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 06 20:26:02 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:02 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:02.243909 27577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:02.234933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:40421)
0806 20:26:02.235498 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:02.235514 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:02.235685 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:02.236007 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:02.236011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:02.236376 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:02.236526 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:02.236534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.237520 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.237524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:02.238720 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:02.238734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.239793 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.239798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:02.239915 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:02.240587 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:02.240604 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:02.243554 (+ 2950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":339,"threads_started":1}
W20260806 20:26:02.244019 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:02.235039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:40421 (local address 127.17.109.254:38721)
0806 20:26:02.235272 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:02.235276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:02.235625 (+ 349us) server_negotiation.cc:407] Connection header received
0806 20:26:02.235794 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:02.235797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:02.235861 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:02.235945 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:02.236648 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.237409 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.238862 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.239605 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.239755 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:02.243780 (+ 4025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:40421: BlockingRecv error: recv got EOF from 127.17.109.194:40421 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 06 20:26:02 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:02 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:02.731690 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:02.714287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:42347)
0806 20:26:02.714884 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:02.714902 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:02.715029 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:02.715496 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:02.715500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:02.716051 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:02.716252 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:02.716288 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.718054 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.718058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:02.719039 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:02.719052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.720211 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.720216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:02.720343 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:02.727948 (+ 7605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:02.727976 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:02.731396 (+ 3420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":311,"threads_started":1}
W20260806 20:26:02.732245 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:02.714375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:42347 (local address 127.17.109.254:38721)
0806 20:26:02.714606 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:02.714615 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:02.714990 (+ 375us) server_negotiation.cc:407] Connection header received
0806 20:26:02.715160 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:02.715168 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:02.715292 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:02.715417 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:02.716463 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.717909 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.719192 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.720069 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.720216 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:02.731741 (+ 11525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:42347: BlockingRecv error: recv got EOF from 127.17.109.195:42347 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 06 20:26:02 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:02 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:02.785778 27579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:02.776897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:47999)
0806 20:26:02.777522 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:02.777543 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:02.777683 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:02.778076 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:02.778081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:02.778558 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:02.778721 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:02.778730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.779808 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.779815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:02.780718 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:02.780728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.781469 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.781475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:02.781654 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:02.782196 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:02.782213 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:02.785417 (+ 3204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":324,"threads_started":1}
W20260806 20:26:02.785832 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:02.776982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:47999 (local address 127.17.109.254:38721)
0806 20:26:02.777343 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:02.777351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:02.777630 (+ 279us) server_negotiation.cc:407] Connection header received
0806 20:26:02.777822 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:02.777829 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:02.777919 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:02.778037 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:02.778887 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.779671 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.780870 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:02.781301 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:02.781439 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:02.785619 (+ 4180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:47999: BlockingRecv error: recv got EOF from 127.17.109.193:47999 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 20:26:03 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:03 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:03.255152 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:03.244920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:38215)
0806 20:26:03.245783 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:03.245802 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:03.245980 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:03.246395 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:03.246404 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:03.246908 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:03.247092 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:03.247102 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.248369 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.248376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:03.249454 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:03.249467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.250390 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.250396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:03.250586 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:03.251211 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:03.251228 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:03.254882 (+ 3654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":692,"thread_start_us":506,"threads_started":1}
W20260806 20:26:03.255450 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:03.245275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:38215 (local address 127.17.109.254:38721)
0806 20:26:03.245623 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:03.245631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:03.245908 (+ 277us) server_negotiation.cc:407] Connection header received
0806 20:26:03.246126 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:03.246132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:03.246233 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:03.246352 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:03.247254 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.248193 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.249758 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.250221 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.250374 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:03.255071 (+ 4697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:38215: BlockingRecv error: recv got EOF from 127.17.109.194:38215 (error 108)
Metrics: {"server-negotiator.queue_time_us":187}
Aug 06 20:26:03 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:03 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:03.742442 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:03.732849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:44305)
0806 20:26:03.733453 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:03.733471 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:03.733654 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:03.734425 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:03.734430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:03.735062 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:03.735325 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:03.735341 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.736388 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.736393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:03.737460 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:03.737474 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.738226 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.738231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:03.738385 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:03.739034 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:03.739053 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:03.742076 (+ 3023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":284,"threads_started":1}
W20260806 20:26:03.742573 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:03.732926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:44305 (local address 127.17.109.254:38721)
0806 20:26:03.733234 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:03.733241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:03.733573 (+ 332us) server_negotiation.cc:407] Connection header received
0806 20:26:03.733878 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:03.733884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:03.733999 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:03.734115 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:03.735502 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.736241 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.737643 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.738079 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.738177 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:03.742288 (+ 4111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:44305: BlockingRecv error: recv got EOF from 127.17.109.195:44305 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Aug 06 20:26:03 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:03 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:03.795663 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:03.786870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:49225)
0806 20:26:03.787429 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:03.787445 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:03.787600 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:03.787941 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:03.787944 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:03.788369 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:03.788520 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:03.788528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.789593 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.789596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:03.790614 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:03.790624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.791382 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.791387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:03.791509 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:03.792126 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:03.792143 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:03.795274 (+ 3131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":285,"threads_started":1}
W20260806 20:26:03.795821 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:03.786929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:49225 (local address 127.17.109.254:38721)
0806 20:26:03.787110 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:03.787114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:03.787521 (+ 407us) server_negotiation.cc:407] Connection header received
0806 20:26:03.787754 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:03.787759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:03.787820 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:03.787903 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:03.788641 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.789470 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.790758 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:03.791230 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:03.791324 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:03.795521 (+ 4197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:49225: BlockingRecv error: recv got EOF from 127.17.109.193:49225 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 06 20:26:04 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:04 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:04.271194 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:04.256306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:55875)
0806 20:26:04.256891 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:04.256908 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:04.257113 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:04.257475 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:04.257480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:04.257907 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:04.258056 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:04.258065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.259092 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.259105 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:04.260279 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:04.260292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.261031 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.261037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:04.261238 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:04.268024 (+ 6786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:04.268043 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:04.270966 (+ 2923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":320,"threads_started":1}
W20260806 20:26:04.271543 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:04.256389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:55875 (local address 127.17.109.254:38721)
0806 20:26:04.256654 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:04.256664 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:04.257031 (+ 367us) server_negotiation.cc:407] Connection header received
0806 20:26:04.257242 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:04.257254 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:04.257317 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:04.257433 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:04.258207 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.258949 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.260424 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.260868 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.261014 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:04.271281 (+ 10267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:55875: BlockingRecv error: recv got EOF from 127.17.109.194:55875 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 06 20:26:04 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:04 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:04.752187 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:04.743350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:41929)
0806 20:26:04.743925 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:04.743942 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:04.744105 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:04.744535 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:04.744539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:04.744963 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:04.745113 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:04.745122 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.746151 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.746157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:04.747063 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:04.747074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.747919 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.747925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:04.748108 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:04.748923 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:04.748941 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:04.751969 (+ 3028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":344,"threads_started":1}
W20260806 20:26:04.752487 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:04.743427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:41929 (local address 127.17.109.254:38721)
0806 20:26:04.743711 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:04.743721 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:04.744034 (+ 313us) server_negotiation.cc:407] Connection header received
0806 20:26:04.744306 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:04.744310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:04.744389 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:04.744499 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:04.745265 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.746001 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.747208 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.747739 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.747890 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:04.752170 (+ 4280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:41929: BlockingRecv error: recv got EOF from 127.17.109.195:41929 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 20:26:04 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:04 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:04.805572 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:04.796870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:43551)
0806 20:26:04.797550 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:04.797566 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:04.797765 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:04.798111 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:04.798116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:04.798592 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:04.798770 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:04.798779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.799875 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.799882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:04.800790 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:04.800801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.801517 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.801523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:04.801708 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:04.802276 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:04.802293 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:04.805293 (+ 3000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":286,"threads_started":1}
W20260806 20:26:04.805748 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:04.797032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:43551 (local address 127.17.109.254:38721)
0806 20:26:04.797266 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:04.797275 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:04.797647 (+ 372us) server_negotiation.cc:407] Connection header received
0806 20:26:04.797899 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:04.797906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:04.797974 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:04.798079 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:04.798924 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.799735 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.800935 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:04.801356 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:04.801488 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:04.805545 (+ 4057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:43551: BlockingRecv error: recv got EOF from 127.17.109.193:43551 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 20:26:05 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:05 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:05.281206 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:05.272213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:33575)
0806 20:26:05.272786 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:05.272803 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:05.272920 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:05.273309 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:05.273317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:05.273927 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:05.274132 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:05.274145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.275392 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.275397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:05.276363 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:05.276377 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.277201 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.277205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:05.277324 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:05.277967 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:05.277986 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:05.280846 (+ 2860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":227,"threads_started":1}
W20260806 20:26:05.281430 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:05.272230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:33575 (local address 127.17.109.254:38721)
0806 20:26:05.272513 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:05.272523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:05.272890 (+ 367us) server_negotiation.cc:407] Connection header received
0806 20:26:05.273018 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:05.273022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:05.273085 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:05.273166 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:05.274313 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.275254 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.276521 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.277057 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.277182 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:05.281106 (+ 3924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:33575: BlockingRecv error: recv got EOF from 127.17.109.194:33575 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 06 20:26:05 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:05 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:05.762223 27587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:05.753162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:58053)
0806 20:26:05.753805 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:05.753821 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:05.753985 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:05.754413 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:05.754418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:05.754928 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:05.755096 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:05.755106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.756311 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.756318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:05.757279 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:05.757291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.758047 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.758051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:05.758174 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:05.758869 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:05.758887 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:05.761979 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":298,"threads_started":1}
W20260806 20:26:05.762408 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:05.753296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:58053 (local address 127.17.109.254:38721)
0806 20:26:05.753537 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:05.753546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:05.753919 (+ 373us) server_negotiation.cc:407] Connection header received
0806 20:26:05.754131 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:05.754135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:05.754238 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:05.754356 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:05.755262 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.756165 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.757426 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.757885 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.758018 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:05.762204 (+ 4186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:58053: BlockingRecv error: recv got EOF from 127.17.109.195:58053 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 20:26:05 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:05 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:05.815414 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:05.806612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:41689)
0806 20:26:05.807322 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:05.807341 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:05.807484 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:05.807900 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:05.807904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:05.808360 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:05.808538 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:05.808549 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.809645 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.809649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:05.810624 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:05.810636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.811356 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.811361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:05.811511 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:05.812221 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:05.812239 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:05.815118 (+ 2879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":365,"threads_started":1}
W20260806 20:26:05.815682 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:05.806817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:41689 (local address 127.17.109.254:38721)
0806 20:26:05.807003 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:05.807008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:05.807427 (+ 419us) server_negotiation.cc:407] Connection header received
0806 20:26:05.807658 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:05.807665 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:05.807759 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:05.807840 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:05.808759 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.809475 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.810757 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:05.811185 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:05.811279 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:05.815381 (+ 4102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:41689: BlockingRecv error: recv got EOF from 127.17.109.193:41689 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 20:26:06 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:06 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:06.291108 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:06.282212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:60435)
0806 20:26:06.282755 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:06.282772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:06.282934 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:06.283284 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:06.283289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:06.283755 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:06.283913 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:06.283921 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.285102 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.285108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:06.286034 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:06.286046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.286822 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.286830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:06.286986 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:06.287643 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:06.287662 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:06.290767 (+ 3105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":273,"threads_started":1}
W20260806 20:26:06.291225 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:06.282307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:60435 (local address 127.17.109.254:38721)
0806 20:26:06.282538 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:06.282547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:06.282863 (+ 316us) server_negotiation.cc:407] Connection header received
0806 20:26:06.283063 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:06.283072 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:06.283142 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:06.283247 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:06.284062 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.284958 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.286178 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.286617 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.286758 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:06.291008 (+ 4250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:60435: BlockingRecv error: recv got EOF from 127.17.109.194:60435 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 20:26:06 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:06 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:06.772061 27590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:06.763197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:53931)
0806 20:26:06.763808 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:06.763827 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:06.763951 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:06.764279 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:06.764283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:06.764682 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:06.764842 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:06.764851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.765898 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.765903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:06.766908 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:06.766920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.767799 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.767804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:06.767924 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:06.768613 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:06.768640 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:06.771802 (+ 3162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":329,"threads_started":1}
W20260806 20:26:06.772251 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:06.763340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:53931 (local address 127.17.109.254:38721)
0806 20:26:06.763641 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:06.763647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:06.763911 (+ 264us) server_negotiation.cc:407] Connection header received
0806 20:26:06.764062 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:06.764066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:06.764128 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:06.764206 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:06.764978 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.765749 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.767049 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.767477 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.767637 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:06.771995 (+ 4358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:53931: BlockingRecv error: recv got EOF from 127.17.109.195:53931 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 06 20:26:06 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:06 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:06.825287 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:06.816532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:60335)
0806 20:26:06.817238 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:06.817255 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:06.817412 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:06.817798 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:06.817803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:06.818334 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:06.818530 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:06.818539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.819542 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.819548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:06.820485 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:06.820496 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.821246 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.821252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:06.821426 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:06.822061 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:06.822079 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:06.825018 (+ 2939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":425,"threads_started":1}
W20260806 20:26:06.825456 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:06.816727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:60335 (local address 127.17.109.254:38721)
0806 20:26:06.816952 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:06.816958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:06.817340 (+ 382us) server_negotiation.cc:407] Connection header received
0806 20:26:06.817542 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:06.817548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:06.817639 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:06.817752 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:06.818671 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.819411 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.820626 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:06.821080 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:06.821220 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:06.825252 (+ 4032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:60335: BlockingRecv error: recv got EOF from 127.17.109.193:60335 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 06 20:26:07 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:07 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:07.301312 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:07.292275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:42957)
0806 20:26:07.293091 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:07.293109 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:07.293269 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:07.293706 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:07.293711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:07.294188 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:07.294362 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:07.294372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.295358 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.295361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:07.296340 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:07.296352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.297026 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.297031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:07.297152 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:07.297725 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:07.297742 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:07.301052 (+ 3310us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":474,"threads_started":1}
W20260806 20:26:07.301558 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:07.292560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:42957 (local address 127.17.109.254:38721)
0806 20:26:07.292749 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:07.292757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:07.293216 (+ 459us) server_negotiation.cc:407] Connection header received
0806 20:26:07.293410 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:07.293414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:07.293511 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:07.293622 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:07.294491 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.295239 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.296472 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.296882 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.296974 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:07.301293 (+ 4319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:42957: BlockingRecv error: recv got EOF from 127.17.109.194:42957 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 20:26:07 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:07 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:07.788992 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:07.773166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:43559)
0806 20:26:07.773841 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:07.773860 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:07.774051 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:07.774459 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:07.774466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:07.774932 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:07.775113 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:07.775145 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.776306 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.776315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:07.777334 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:07.777347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.778154 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.778161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:07.778448 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:07.785745 (+ 7297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:07.785764 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:07.788743 (+ 2979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":378,"threads_started":1}
W20260806 20:26:07.789243 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:07.773306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:43559 (local address 127.17.109.254:38721)
0806 20:26:07.773574 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:07.773584 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:07.773959 (+ 375us) server_negotiation.cc:407] Connection header received
0806 20:26:07.774208 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:07.774221 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:07.774290 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:07.774407 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:07.775308 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.776161 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.777495 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.777967 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.778120 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:07.788976 (+ 10856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:43559: BlockingRecv error: recv got EOF from 127.17.109.195:43559 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 06 20:26:07 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:07 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:07.835525 27595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:07.826520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:33727)
0806 20:26:07.827164 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:07.827203 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:07.827352 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:07.827745 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:07.827751 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:07.828251 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:07.828438 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:07.828449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.829597 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.829605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:07.830602 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:07.830613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.831368 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.831374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:07.831550 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:07.832260 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:07.832277 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:07.835242 (+ 2965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":410,"threads_started":1}
W20260806 20:26:07.835744 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:07.826574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:33727 (local address 127.17.109.254:38721)
0806 20:26:07.826853 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:07.826863 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:07.827314 (+ 451us) server_negotiation.cc:407] Connection header received
0806 20:26:07.827509 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:07.827513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:07.827592 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:07.827696 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:07.828590 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.829475 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.830769 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:07.831211 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:07.831356 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:07.835452 (+ 4096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:33727: BlockingRecv error: recv got EOF from 127.17.109.193:33727 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 20:26:08 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:08 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:08.310672 27597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:08.302426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:40167)
0806 20:26:08.302953 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:08.302969 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:08.303078 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:08.303365 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:08.303368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:08.303818 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:08.304003 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:08.304018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.305126 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.305130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:08.306120 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:08.306132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.306773 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.306777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:08.306888 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:08.307441 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:08.307458 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:08.310425 (+ 2967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":298,"threads_started":1}
W20260806 20:26:08.310869 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:08.302486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:40167 (local address 127.17.109.254:38721)
0806 20:26:08.302652 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:08.302656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:08.303049 (+ 393us) server_negotiation.cc:407] Connection header received
0806 20:26:08.303162 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:08.303166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:08.303236 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:08.303320 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:08.304136 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.304972 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.306249 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.306632 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.306735 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:08.310629 (+ 3894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:40167: BlockingRecv error: recv got EOF from 127.17.109.194:40167 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 20:26:08 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:08 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:08.799127 27598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:08.790069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:52541)
0806 20:26:08.790611 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:08.790628 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:08.790760 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:08.791085 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:08.791088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:08.791515 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:08.791748 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:08.791758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.792888 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.792894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:08.793925 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:08.793940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.794955 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.794960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:08.795078 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:08.795768 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:08.795795 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:08.798899 (+ 3104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":315,"threads_started":1}
W20260806 20:26:08.799391 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:08.790339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:52541 (local address 127.17.109.254:38721)
0806 20:26:08.790575 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:08.790583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:08.790694 (+ 111us) server_negotiation.cc:407] Connection header received
0806 20:26:08.790890 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:08.790894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:08.790958 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:08.791043 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:08.791892 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.792740 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.794090 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.794744 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.794879 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:08.799116 (+ 4237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:52541: BlockingRecv error: recv got EOF from 127.17.109.195:52541 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 06 20:26:08 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:08 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:08.845367 27599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:08.836447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:55043)
0806 20:26:08.837024 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:08.837041 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:08.837157 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:08.837479 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:08.837482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:08.837928 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:08.838111 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:08.838119 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.839183 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.839188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:08.840270 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:08.840282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.841010 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.841015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:08.841143 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:08.841779 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:08.841796 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:08.845109 (+ 3313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":344,"threads_started":1}
W20260806 20:26:08.845646 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:08.836622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:55043 (local address 127.17.109.254:38721)
0806 20:26:08.836871 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:08.836877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:08.837121 (+ 244us) server_negotiation.cc:407] Connection header received
0806 20:26:08.837283 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:08.837287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:08.837352 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:08.837437 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:08.838268 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.839048 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.840396 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:08.840831 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:08.840929 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:08.845377 (+ 4448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:55043: BlockingRecv error: recv got EOF from 127.17.109.193:55043 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 06 20:26:09 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:09 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:09.320734 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:09.311627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:32983)
0806 20:26:09.312167 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:09.312183 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:09.312297 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:09.312583 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:09.312586 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:09.313050 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:09.313225 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:09.313234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.314239 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.314243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:09.315136 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:09.315147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.315930 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.315935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:09.316060 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:09.316973 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:09.316990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:09.320397 (+ 3407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":301,"threads_started":1}
W20260806 20:26:09.320979 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:09.311770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:32983 (local address 127.17.109.254:38721)
0806 20:26:09.311957 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:09.311962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:09.312262 (+ 300us) server_negotiation.cc:407] Connection header received
0806 20:26:09.312391 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:09.312395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:09.312457 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:09.312540 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:09.313352 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.314105 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.315262 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.315785 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.315908 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:09.320659 (+ 4751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:32983: BlockingRecv error: recv got EOF from 127.17.109.194:32983 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 20:26:09 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:09 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:09.809592 27601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:09.800212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:34039)
0806 20:26:09.800808 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:09.800824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:09.800943 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:09.801266 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:09.801269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:09.801721 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:09.801901 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:09.801910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.803064 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.803068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:09.804175 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:09.804188 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.805130 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.805137 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:09.805353 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:09.806020 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:09.806039 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:09.809170 (+ 3131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":343,"threads_started":1}
W20260806 20:26:09.809686 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:09.800382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:34039 (local address 127.17.109.254:38721)
0806 20:26:09.800575 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:09.800580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:09.800916 (+ 336us) server_negotiation.cc:407] Connection header received
0806 20:26:09.801054 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:09.801058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:09.801128 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:09.801229 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:09.802075 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.802925 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.804290 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.804931 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.805095 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:09.809403 (+ 4308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:34039: BlockingRecv error: recv got EOF from 127.17.109.195:34039 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 20:26:09 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:09 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:09.855065 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:09.846399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:56773)
0806 20:26:09.846966 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:09.846982 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:09.847153 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:09.847494 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:09.847499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:09.848014 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:09.848188 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:09.848197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.849249 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.849255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:09.850195 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:09.850206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.850987 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.850993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:09.851186 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:09.851821 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:09.851838 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:09.854723 (+ 2885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":325,"threads_started":1}
W20260806 20:26:09.855185 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:09.846451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:56773 (local address 127.17.109.254:38721)
0806 20:26:09.846770 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:09.846779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:09.847073 (+ 294us) server_negotiation.cc:407] Connection header received
0806 20:26:09.847280 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:09.847285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:09.847360 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:09.847463 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:09.848336 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.849104 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.850333 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:09.850835 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:09.850977 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:09.854937 (+ 3960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:56773: BlockingRecv error: recv got EOF from 127.17.109.193:56773 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 20:26:10 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:10 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:10.331513 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:10.322081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:36121)
0806 20:26:10.322705 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:10.322722 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:10.322838 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:10.323234 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:10.323241 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:10.323813 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:10.323976 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:10.323986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.325083 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.325088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:10.326169 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:10.326184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.326959 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.326963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:10.327088 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:10.327747 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:10.327764 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:10.331174 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":386,"threads_started":1}
W20260806 20:26:10.331682 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:10.322175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:36121 (local address 127.17.109.254:38721)
0806 20:26:10.322395 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:10.322400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:10.322837 (+ 437us) server_negotiation.cc:407] Connection header received
0806 20:26:10.322928 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:10.322932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:10.323003 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:10.323098 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:10.324140 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.324888 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.326322 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.326824 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.326934 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:10.331423 (+ 4489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:36121: BlockingRecv error: recv got EOF from 127.17.109.194:36121 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 20:26:10 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:10 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:10.819460 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:10.810622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:56063)
0806 20:26:10.811162 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:10.811179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:10.811299 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:10.811657 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:10.811660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:10.812087 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:10.812246 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:10.812255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.813297 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.813302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:10.814421 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:10.814435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.815128 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.815133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:10.815278 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:10.815920 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:10.815940 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:10.819199 (+ 3259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":331,"threads_started":1}
W20260806 20:26:10.819831 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:10.810687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:56063 (local address 127.17.109.254:38721)
0806 20:26:10.810922 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:10.810931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:10.811272 (+ 341us) server_negotiation.cc:407] Connection header received
0806 20:26:10.811402 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:10.811406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:10.811469 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:10.811592 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:10.812382 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.813124 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.814571 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.814994 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.815087 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:10.819471 (+ 4384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:56063: BlockingRecv error: recv got EOF from 127.17.109.195:56063 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 06 20:26:10 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:10 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:10.865823 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:10.856277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:56777)
0806 20:26:10.856860 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:10.856876 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:10.857041 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:10.857402 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:10.857406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:10.857863 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:10.858020 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:10.858029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.859143 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.859148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:10.860815 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:10.860828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.861601 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.861607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:10.861791 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:10.862408 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:10.862426 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:10.865384 (+ 2958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":276,"threads_started":1}
W20260806 20:26:10.865835 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:10.856378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:56777 (local address 127.17.109.254:38721)
0806 20:26:10.856676 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:10.856682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:10.856968 (+ 286us) server_negotiation.cc:407] Connection header received
0806 20:26:10.857174 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:10.857178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:10.857260 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:10.857367 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:10.858169 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.858978 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.860957 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:10.861450 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:10.861590 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:10.865597 (+ 4007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:56777: BlockingRecv error: recv got EOF from 127.17.109.193:56777 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 20:26:11 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:11 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:11.342675 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:11.332633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:48127)
0806 20:26:11.333296 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:11.333322 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:11.333474 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:11.333944 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:11.333948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:11.334483 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:11.334720 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:11.334734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.336005 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.336010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:11.337207 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:11.337223 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.338074 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.338080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:11.338248 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:11.338921 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:11.338942 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:11.342382 (+ 3440us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":341,"threads_started":1}
W20260806 20:26:11.342846 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:11.332760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:48127 (local address 127.17.109.254:38721)
0806 20:26:11.333012 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:11.333020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:11.333419 (+ 399us) server_negotiation.cc:407] Connection header received
0806 20:26:11.333645 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:11.333652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:11.333756 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:11.333887 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:11.334928 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.335847 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.337368 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.337796 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.337903 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:11.342601 (+ 4698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:48127: BlockingRecv error: recv got EOF from 127.17.109.194:48127 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 20:26:11 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:11 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:11.829993 27607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:11.820712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:43271)
0806 20:26:11.821243 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:11.821260 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:11.821428 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:11.821838 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:11.821843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:11.822291 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:11.822445 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:11.822454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.823628 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.823634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:11.824906 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:11.824919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.825636 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.825642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:11.825812 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:11.826435 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:11.826453 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:11.829686 (+ 3233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":272,"threads_started":1}
W20260806 20:26:11.830193 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:11.820698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:43271 (local address 127.17.109.254:38721)
0806 20:26:11.820956 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:11.820964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:11.821356 (+ 392us) server_negotiation.cc:407] Connection header received
0806 20:26:11.821568 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:11.821573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:11.821649 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:11.821801 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:11.822593 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.823460 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.825042 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.825454 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.825595 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:11.829996 (+ 4401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:43271: BlockingRecv error: recv got EOF from 127.17.109.195:43271 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 20:26:11 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:11 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:11.875700 27608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:11.866980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:53019)
0806 20:26:11.867596 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:11.867613 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:11.867734 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:11.868052 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:11.868055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:11.868428 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:11.868605 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:11.868613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.869633 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.869637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:11.870563 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:11.870573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.871253 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.871259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:11.871438 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:11.872257 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:11.872275 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:11.875409 (+ 3134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":340,"threads_started":1}
W20260806 20:26:11.876180 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:11.867127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:53019 (local address 127.17.109.254:38721)
0806 20:26:11.867386 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:11.867391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:11.867689 (+ 298us) server_negotiation.cc:407] Connection header received
0806 20:26:11.867853 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:11.867856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:11.867923 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:11.868006 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:11.868737 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.869499 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.870686 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:11.871094 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:11.871187 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:11.875766 (+ 4579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:53019: BlockingRecv error: recv got EOF from 127.17.109.193:53019 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 06 20:26:12 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:12 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:12.352759 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:12.344017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:50033)
0806 20:26:12.344574 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:12.344598 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:12.344724 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:12.345045 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:12.345048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:12.345461 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:12.345645 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:12.345654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.346725 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.346729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:12.347733 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:12.347750 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.348729 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.348734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:12.348896 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:12.349648 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:12.349672 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:12.352506 (+ 2834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":319,"threads_started":1}
W20260806 20:26:12.352975 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:12.344127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:50033 (local address 127.17.109.254:38721)
0806 20:26:12.344353 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:12.344358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:12.344744 (+ 386us) server_negotiation.cc:407] Connection header received
0806 20:26:12.344841 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:12.344845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:12.344914 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:12.344998 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:12.345807 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.346582 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.347924 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.348579 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.348779 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:12.352733 (+ 3954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:50033: BlockingRecv error: recv got EOF from 127.17.109.194:50033 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 06 20:26:12 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:12 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.195@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:12.845330 27610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:12.831112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.195:40711)
0806 20:26:12.831771 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:12.831790 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:12.831919 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:12.832274 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:12.832278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:12.832669 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:12.832912 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:12.832922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.833998 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.834002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:12.835031 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:12.835044 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.835788 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.835793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:12.835919 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:12.842047 (+ 6128us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:12.842068 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:12.845053 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":325,"threads_started":1}
W20260806 20:26:12.845510 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:12.831258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:40711 (local address 127.17.109.254:38721)
0806 20:26:12.831523 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:12.831532 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:12.831881 (+ 349us) server_negotiation.cc:407] Connection header received
0806 20:26:12.832030 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:12.832034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:12.832102 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:12.832196 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:12.833051 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.833857 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.835156 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.835615 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.835708 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:12.845290 (+ 9582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.195:40711: BlockingRecv error: recv got EOF from 127.17.109.195:40711 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 06 20:26:12 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:12 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.193@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:12.885604 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:12.876825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.193:48899)
0806 20:26:12.877390 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:12.877411 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:12.877555 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:12.877920 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:12.877925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:12.878362 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:12.878535 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:12.878544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.879723 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.879729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:12.880695 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:12.880707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.881449 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.881455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:12.881668 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:12.882308 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:12.882325 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:12.885236 (+ 2911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":293,"threads_started":1}
W20260806 20:26:12.885717 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:12.877124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:48899 (local address 127.17.109.254:38721)
0806 20:26:12.877382 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:12.877388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:12.877498 (+ 110us) server_negotiation.cc:407] Connection header received
0806 20:26:12.877700 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:12.877709 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:12.877771 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:12.877873 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:12.878731 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.879515 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.880844 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:12.881277 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:12.881421 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:12.885454 (+ 4033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.193:48899: BlockingRecv error: recv got EOF from 127.17.109.193:48899 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 06 20:26:13 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 20:26:13 dist-test-slave-6k22 krb5kdc[26502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.109.194@KRBTEST.COM for oryx/127.17.109.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 20:26:13.362600 27612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:13.353888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38721 (local address 127.17.109.194:48817)
0806 20:26:13.354486 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:13.354503 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:13.354614 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:13.354907 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:13.354910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:13.355322 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:13.355494 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:13.355503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.356617 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.356621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:13.357589 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:13.357601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.358439 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.358444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:13.358564 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:13.359266 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:13.359283 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:13.362196 (+ 2913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.109.254:38721: Server oryx/127.17.109.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":341,"threads_started":1}
W20260806 20:26:13.362658 27286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:13.353994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:48817 (local address 127.17.109.254:38721)
0806 20:26:13.354160 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:13.354165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:13.354578 (+ 413us) server_negotiation.cc:407] Connection header received
0806 20:26:13.354697 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:13.354700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:13.354767 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:13.354848 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:13.355673 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.356461 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.357783 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.358278 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.358411 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:13.362405 (+ 3994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.109.194:48817: BlockingRecv error: recv got EOF from 127.17.109.194:48817 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 20:26:13.645013 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27129
I20260806 20:26:13.658658 27614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:13.646217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.17.109.193:39597)
0806 20:26:13.646869 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:13.646877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:13.646906 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:26:13.646976 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:13.646982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:13.647215 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:13.647379 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:13.648892 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.650043 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.651305 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.652054 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.652167 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:13.655392 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:13.655404 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:13.655418 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:13.655456 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:13.658135 (+ 2679us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.109.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 20:26:13.658421 (+ 286us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41112: Request ticket server kudu/127.17.109.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":332,"threads_started":1}
W20260806 20:26:13.658828 27614 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41112: Request ticket server kudu/127.17.109.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 20:26:13.659029 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27129: Not authorized: Client connection negotiation failed: client connection to 127.17.109.193:39597: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.109.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 20:26:13.659308 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 27129
I20260806 20:26:13.681613 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27288
I20260806 20:26:13.695302 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:13.682905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.17.109.194:32779)
0806 20:26:13.683697 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:13.683710 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:13.683756 (+ 46us) server_negotiation.cc:407] Connection header received
0806 20:26:13.683832 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:13.683839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:13.684083 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:13.684270 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:13.685650 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.686953 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.688374 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.688952 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.689069 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:13.692044 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:13.692056 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:13.692075 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:13.692127 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:13.694795 (+ 2668us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.109.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 20:26:13.695054 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40978: Request ticket server kudu/127.17.109.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":364,"threads_started":1}
W20260806 20:26:13.695427 27615 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40978: Request ticket server kudu/127.17.109.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 20:26:13.695425 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27288: Not authorized: Client connection negotiation failed: client connection to 127.17.109.194:32779: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.109.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 20:26:13.695708 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 27288
I20260806 20:26:13.715114 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27426
I20260806 20:26:13.727800 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:13.716248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45622 (local address 127.17.109.195:33713)
0806 20:26:13.716833 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:13.716840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:13.716867 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:26:13.716928 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:13.716934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:13.717162 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:13.717372 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:13.718759 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.719938 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.721134 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:13.721695 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:13.721841 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:13.724518 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:13.724527 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:13.724540 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:13.724602 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:13.727095 (+ 2493us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.109.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0806 20:26:13.727428 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45622: Request ticket server kudu/127.17.109.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":299,"threads_started":1}
W20260806 20:26:13.727923 27616 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45622: Request ticket server kudu/127.17.109.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260806 20:26:13.727869 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27426: Not authorized: Client connection negotiation failed: client connection to 127.17.109.195:33713: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.17.109.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260806 20:26:13.728058 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 27426
I20260806 20:26:13.747798 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27057
I20260806 20:26:13.749267 27122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:13.871049 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 27057
2026-08-06T20:26:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21426 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T20:26:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:26:13Z Disabled control of system clock
I20260806 20:26:13.962175 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:42793
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:42793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 20:26:14.310892 27621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:14.311187 27621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:14.311357 27621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:14.319360 27621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:26:14.319468 27621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:14.319533 27621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:14.319645 27621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:26:14.319698 27621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:26:14.328987 27621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:42793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:42793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:14.330381 27621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:14.332275 27621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:14.345927 27628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:14.346547 27626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:14.346601 27630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:14.348807 27621 server_base.cc:1034] running on GCE node
I20260806 20:26:14.349958 27621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:14.351714 27621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:14.352993 27621 hybrid_clock.cc:648] HybridClock initialized: now 1786047974352923 us; error 58 us; skew 500 ppm
I20260806 20:26:14.353488 27621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:26:14.356386 27621 webserver.cc:460] Webserver started at http://127.17.109.254:42879/ using document root <none> and password file <none>
I20260806 20:26:14.357439 27621 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:14.357574 27621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:14.358035 27621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:14.362252 27621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "978215a5ee37443d87d78424ffae0010"
format_stamp: "Formatted at 2026-08-06 20:26:14 on dist-test-slave-6k22"
server_key: "a24571e5a0b4a639aae35d64d9be064c"
server_key_iv: "cdf6d7eb7d5cabf15f3bbe15d2684179"
server_key_version: "encryptionkey@0"
I20260806 20:26:14.363379 27621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "978215a5ee37443d87d78424ffae0010"
format_stamp: "Formatted at 2026-08-06 20:26:14 on dist-test-slave-6k22"
server_key: "a24571e5a0b4a639aae35d64d9be064c"
server_key_iv: "cdf6d7eb7d5cabf15f3bbe15d2684179"
server_key_version: "encryptionkey@0"
I20260806 20:26:14.370889 27621 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260806 20:26:14.376295 27637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:14.378407 27621 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 20:26:14.378619 27621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "978215a5ee37443d87d78424ffae0010"
format_stamp: "Formatted at 2026-08-06 20:26:14 on dist-test-slave-6k22"
server_key: "a24571e5a0b4a639aae35d64d9be064c"
server_key_iv: "cdf6d7eb7d5cabf15f3bbe15d2684179"
server_key_version: "encryptionkey@0"
I20260806 20:26:14.378865 27621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:14.396588 27621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:14.397478 27621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:14.397806 27621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:14.417407 27621 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:42793
I20260806 20:26:14.417431 27690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:42793 every 8 connection(s)
I20260806 20:26:14.418956 27621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:26:14.420931 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 27621
I20260806 20:26:14.421175 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:26:14.422128 17847 external_mini_cluster.cc:1428] Setting key 886f5bcf8a9e8c1380c9774ef3942c66
I20260806 20:26:14.424332 27691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:26:14.437086 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:14.424368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35574 (local address 127.17.109.254:42793)
0806 20:26:14.425449 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:14.425460 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:14.425503 (+ 43us) server_negotiation.cc:407] Connection header received
0806 20:26:14.427069 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:14.427090 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:14.427615 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:14.427991 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:14.428792 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:14.430152 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:14.432245 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:14.432959 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:14.433087 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:14.434479 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:14.434521 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:14.434547 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:14.435220 (+ 673us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:14.435298 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:14.435319 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:14.435434 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:14.435770 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 20:26:14.436133 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":401,"threads_started":1}
I20260806 20:26:14.438503 27691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010: Bootstrap starting.
I20260806 20:26:14.441847 27691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010: Neither blocks nor log segments found. Creating new log.
I20260806 20:26:14.442926 27691 log.cc:824] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010: Log is configured to *not* fsync() on all Append() calls
I20260806 20:26:14.446432 27691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010: No bootstrap required, opened a new log
I20260806 20:26:14.455112 27691 raft_consensus.cc:348] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978215a5ee37443d87d78424ffae0010" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42793 } }
I20260806 20:26:14.455569 27691 raft_consensus.cc:374] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:26:14.455762 27691 raft_consensus.cc:729] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978215a5ee37443d87d78424ffae0010, State: Initialized, Role: FOLLOWER
I20260806 20:26:14.456458 27691 consensus_queue.cc:260] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978215a5ee37443d87d78424ffae0010" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42793 } }
I20260806 20:26:14.456763 27691 raft_consensus.cc:388] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:26:14.456960 27691 raft_consensus.cc:482] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:26:14.457175 27691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:26:14.459887 27691 raft_consensus.cc:504] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978215a5ee37443d87d78424ffae0010" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42793 } }
I20260806 20:26:14.460479 27691 leader_election.cc:302] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 978215a5ee37443d87d78424ffae0010; no voters:
I20260806 20:26:14.461171 27691 leader_election.cc:288] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:26:14.461326 27696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:26:14.462149 27696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [term 1 LEADER]: Becoming Leader. State: Replica: 978215a5ee37443d87d78424ffae0010, State: Running, Role: LEADER
I20260806 20:26:14.462762 27696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978215a5ee37443d87d78424ffae0010" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42793 } }
I20260806 20:26:14.463985 27691 sys_catalog.cc:564] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:26:14.465946 27697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "978215a5ee37443d87d78424ffae0010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978215a5ee37443d87d78424ffae0010" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42793 } } }
I20260806 20:26:14.465965 27698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 978215a5ee37443d87d78424ffae0010. Latest consensus state: current_term: 1 leader_uuid: "978215a5ee37443d87d78424ffae0010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "978215a5ee37443d87d78424ffae0010" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42793 } } }
I20260806 20:26:14.466599 27697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [sys.catalog]: This master's current role is: LEADER
I20260806 20:26:14.466595 27698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010 [sys.catalog]: This master's current role is: LEADER
I20260806 20:26:14.472718 27702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:26:14.478223 27702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:26:14.487730 27702 catalog_manager.cc:1347] Generated new cluster ID: af86d8f31ea84e14bef6da47f1a762e7
I20260806 20:26:14.487856 27702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:26:14.506877 27702 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:26:14.508307 27702 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:26:14.521384 27702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 978215a5ee37443d87d78424ffae0010: Generated new TSK 0
I20260806 20:26:14.522547 27702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:26:14.537240 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42793
--builtin_ntp_servers=127.17.109.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 20:26:14.906116 27715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:14.906471 27715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:14.906566 27715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:14.914135 27715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:14.914260 27715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:14.914376 27715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:26:14.923870 27715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:14.925202 27715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:14.927177 27715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:14.940755 27723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:14.941005 27727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:14.941069 27724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:14.942165 27715 server_base.cc:1034] running on GCE node
I20260806 20:26:14.942835 27715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:14.943938 27715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:14.945158 27715 hybrid_clock.cc:648] HybridClock initialized: now 1786047974945107 us; error 41 us; skew 500 ppm
I20260806 20:26:14.945550 27715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:26:14.948287 27715 webserver.cc:460] Webserver started at http://127.17.109.193:38533/ using document root <none> and password file <none>
I20260806 20:26:14.949038 27715 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:14.949172 27715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:14.949551 27715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:14.952680 27715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "6001b76206154df2930cffa71bbc445b"
format_stamp: "Formatted at 2026-08-06 20:26:14 on dist-test-slave-6k22"
server_key: "f0d93b515255b477a8415c811901896c"
server_key_iv: "b535ccd7cc8c4dbeda442e5881334f6c"
server_key_version: "encryptionkey@0"
I20260806 20:26:14.953557 27715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "6001b76206154df2930cffa71bbc445b"
format_stamp: "Formatted at 2026-08-06 20:26:14 on dist-test-slave-6k22"
server_key: "f0d93b515255b477a8415c811901896c"
server_key_iv: "b535ccd7cc8c4dbeda442e5881334f6c"
server_key_version: "encryptionkey@0"
I20260806 20:26:14.959275 27715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:26:14.962968 27733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:14.964556 27715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:26:14.964757 27715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "6001b76206154df2930cffa71bbc445b"
format_stamp: "Formatted at 2026-08-06 20:26:14 on dist-test-slave-6k22"
server_key: "f0d93b515255b477a8415c811901896c"
server_key_iv: "b535ccd7cc8c4dbeda442e5881334f6c"
server_key_version: "encryptionkey@0"
I20260806 20:26:14.965005 27715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:14.984818 27715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:14.985843 27715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:14.986222 27715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:14.987283 27715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:14.988931 27715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:14.989064 27715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:14.989228 27715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:14.989336 27715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:15.026988 27715 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:45497
I20260806 20:26:15.027019 27852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:45497 every 8 connection(s)
I20260806 20:26:15.028681 27715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:26:15.037415 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 27715
I20260806 20:26:15.037659 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:26:15.038326 17847 external_mini_cluster.cc:1428] Setting key daf3117b787f9e5d826b76ab332ba346
I20260806 20:26:15.041678 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42793
--builtin_ntp_servers=127.17.109.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 20:26:15.042914 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:15.031463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:50229 (local address 127.17.109.254:42793)
0806 20:26:15.032073 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:15.032078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:15.033104 (+ 1026us) server_negotiation.cc:407] Connection header received
0806 20:26:15.035032 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:15.035038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:15.035139 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:15.035279 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:15.036582 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.037401 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.039521 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.040297 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.040454 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:15.042103 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:15.042109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:15.042112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:15.042131 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:15.042180 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:15.042193 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:15.042274 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:15.042541 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 20:26:15.042626 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":508,"thread_start_us":374,"threads_started":1}
I20260806 20:26:15.044481 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:15.031990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:42793 (local address 127.17.109.193:50229)
0806 20:26:15.032939 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:15.032978 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:15.034741 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:15.035516 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:15.035525 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:15.035703 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:15.036399 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:15.036417 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.037578 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.037584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:15.039287 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:15.039306 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.040548 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.040555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:15.040871 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:15.041648 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:26:15.041653 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:15.041710 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:26:15.041874 (+ 164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:15.042300 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:15.042313 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:15.042325 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 20:26:15.042521 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 20:26:15.042978 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":279,"threads_started":1}
I20260806 20:26:15.046131 27854 heartbeater.cc:347] Connected to a master server at 127.17.109.254:42793
I20260806 20:26:15.046480 27854 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:15.047322 27854 heartbeater.cc:511] Master 127.17.109.254:42793 requested a full tablet report, sending...
I20260806 20:26:15.049518 27655 ts_manager.cc:194] Registered new tserver with Master: 6001b76206154df2930cffa71bbc445b (127.17.109.193:45497)
I20260806 20:26:15.051107 27655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.193:50229
W20260806 20:26:15.378041 27859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:15.378405 27859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:15.378574 27859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:15.386055 27859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:15.386204 27859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:15.386382 27859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:26:15.395753 27859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:15.397292 27859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:15.399107 27859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:15.412532 27865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:15.412631 27864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:15.412609 27867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:15.413625 27859 server_base.cc:1034] running on GCE node
I20260806 20:26:15.414248 27859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:15.415181 27859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:15.416417 27859 hybrid_clock.cc:648] HybridClock initialized: now 1786047975416356 us; error 54 us; skew 500 ppm
I20260806 20:26:15.416838 27859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:26:15.419638 27859 webserver.cc:460] Webserver started at http://127.17.109.194:44259/ using document root <none> and password file <none>
I20260806 20:26:15.420468 27859 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:15.420586 27859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:15.421000 27859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:15.424741 27859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "4e25ebaa8c4148e1975a5c6b5e5977dd"
format_stamp: "Formatted at 2026-08-06 20:26:15 on dist-test-slave-6k22"
server_key: "54b1ee9fd8b61a817d24d60782aea21b"
server_key_iv: "1a4bb08f297addb3cd11afec15ef7594"
server_key_version: "encryptionkey@0"
I20260806 20:26:15.425681 27859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "4e25ebaa8c4148e1975a5c6b5e5977dd"
format_stamp: "Formatted at 2026-08-06 20:26:15 on dist-test-slave-6k22"
server_key: "54b1ee9fd8b61a817d24d60782aea21b"
server_key_iv: "1a4bb08f297addb3cd11afec15ef7594"
server_key_version: "encryptionkey@0"
I20260806 20:26:15.432104 27859 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 20:26:15.436816 27873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:15.438478 27859 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 20:26:15.438634 27859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "4e25ebaa8c4148e1975a5c6b5e5977dd"
format_stamp: "Formatted at 2026-08-06 20:26:15 on dist-test-slave-6k22"
server_key: "54b1ee9fd8b61a817d24d60782aea21b"
server_key_iv: "1a4bb08f297addb3cd11afec15ef7594"
server_key_version: "encryptionkey@0"
I20260806 20:26:15.438926 27859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:15.454664 27859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:15.455443 27859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:15.455789 27859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:15.456697 27859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:15.458184 27859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:15.458273 27859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:15.458348 27859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:15.458424 27859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:15.495200 27859 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:38093
I20260806 20:26:15.495311 27993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:38093 every 8 connection(s)
I20260806 20:26:15.496582 27859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:26:15.501796 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 27859
I20260806 20:26:15.501992 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:26:15.502506 17847 external_mini_cluster.cc:1428] Setting key 7e9bc4b5f29c30ab570efc2da8848831
I20260806 20:26:15.506130 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42793
--builtin_ntp_servers=127.17.109.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 20:26:15.512190 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:15.499377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:59389 (local address 127.17.109.254:42793)
0806 20:26:15.499688 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:15.499694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:15.501239 (+ 1545us) server_negotiation.cc:407] Connection header received
0806 20:26:15.503484 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:15.503493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:15.503621 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:15.503803 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:15.505379 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.506824 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.509082 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.509551 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.509696 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:15.511240 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:15.511247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:15.511252 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:15.511280 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:15.511314 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:15.511327 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:15.511430 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:15.511740 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 20:26:15.511852 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260806 20:26:15.513346 27996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:15.500065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:42793 (local address 127.17.109.194:59389)
0806 20:26:15.501050 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:15.501090 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:15.503133 (+ 2043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:15.504096 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:15.504109 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:15.504233 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:15.505171 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:15.505203 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.507034 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.507043 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:15.508921 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:15.508935 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.509821 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.509832 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:15.510075 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:15.510869 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:26:15.510872 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:15.510924 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:26:15.511009 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:15.511445 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:15.511459 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:15.511467 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 20:26:15.511738 (+ 271us) client_negotiation.cc:240] Negotiation successful
0806 20:26:15.512182 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":311,"threads_started":1}
I20260806 20:26:15.514943 27994 heartbeater.cc:347] Connected to a master server at 127.17.109.254:42793
I20260806 20:26:15.515276 27994 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:15.516171 27994 heartbeater.cc:511] Master 127.17.109.254:42793 requested a full tablet report, sending...
I20260806 20:26:15.517861 27655 ts_manager.cc:194] Registered new tserver with Master: 4e25ebaa8c4148e1975a5c6b5e5977dd (127.17.109.194:38093)
I20260806 20:26:15.518937 27655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.194:59389
W20260806 20:26:15.843988 27997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:15.844256 27997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:15.844417 27997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:15.851456 27997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:15.851549 27997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:15.851769 27997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:26:15.860708 27997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42793
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:15.861990 27997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:15.863811 27997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:15.877679 28003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:15.877799 28002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:15.877707 28005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:15.878770 27997 server_base.cc:1034] running on GCE node
I20260806 20:26:15.879501 27997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:15.880654 27997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:15.881922 27997 hybrid_clock.cc:648] HybridClock initialized: now 1786047975881863 us; error 67 us; skew 500 ppm
I20260806 20:26:15.882288 27997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:26:15.885018 27997 webserver.cc:460] Webserver started at http://127.17.109.195:39515/ using document root <none> and password file <none>
I20260806 20:26:15.885807 27997 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:15.885902 27997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:15.886279 27997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:15.889252 27997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "34e26848800a424d93565886c32903f1"
format_stamp: "Formatted at 2026-08-06 20:26:15 on dist-test-slave-6k22"
server_key: "90c278997b46abf1137ad41e98afbafc"
server_key_iv: "521b5727da5312b3a508fca1536067c5"
server_key_version: "encryptionkey@0"
I20260806 20:26:15.890038 27997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "34e26848800a424d93565886c32903f1"
format_stamp: "Formatted at 2026-08-06 20:26:15 on dist-test-slave-6k22"
server_key: "90c278997b46abf1137ad41e98afbafc"
server_key_iv: "521b5727da5312b3a508fca1536067c5"
server_key_version: "encryptionkey@0"
I20260806 20:26:15.895157 27997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 20:26:15.899250 28011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:15.900718 27997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:26:15.900882 27997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "34e26848800a424d93565886c32903f1"
format_stamp: "Formatted at 2026-08-06 20:26:15 on dist-test-slave-6k22"
server_key: "90c278997b46abf1137ad41e98afbafc"
server_key_iv: "521b5727da5312b3a508fca1536067c5"
server_key_version: "encryptionkey@0"
I20260806 20:26:15.901103 27997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:15.926728 27997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:15.927683 27997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:15.928023 27997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:15.929078 27997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:15.930583 27997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:15.930706 27997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:15.930814 27997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:15.930907 27997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:15.966544 27997 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:40229
I20260806 20:26:15.966742 28130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:40229 every 8 connection(s)
I20260806 20:26:15.968042 27997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:26:15.968433 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 27997
I20260806 20:26:15.968663 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:26:15.969233 17847 external_mini_cluster.cc:1428] Setting key bae852b3516c81db3950fe34b28590d6
I20260806 20:26:15.981906 27856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:15.971187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:58005 (local address 127.17.109.254:42793)
0806 20:26:15.971468 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:15.971474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:15.973165 (+ 1691us) server_negotiation.cc:407] Connection header received
0806 20:26:15.975084 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:15.975091 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:15.975175 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:15.975274 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:15.976574 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.977480 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.979114 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.979551 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.979759 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:15.981174 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:15.981179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:15.981183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:15.981202 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:15.981240 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:15.981252 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:15.981329 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:15.981535 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 20:26:15.981631 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260806 20:26:15.983088 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:15.971839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:42793 (local address 127.17.109.195:58005)
0806 20:26:15.972990 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:15.973047 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:15.974803 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:15.975545 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:15.975615 (+ 70us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:15.975772 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:15.976386 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:15.976405 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.977652 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.977657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:15.978972 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:15.978985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:15.979865 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:15.979870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:15.980077 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:15.980843 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:26:15.980846 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:15.980889 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:26:15.980976 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:15.981341 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:15.981349 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:15.981355 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 20:26:15.981504 (+ 149us) client_negotiation.cc:240] Negotiation successful
0806 20:26:15.981888 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":405,"threads_started":1}
I20260806 20:26:15.984994 28131 heartbeater.cc:347] Connected to a master server at 127.17.109.254:42793
I20260806 20:26:15.985356 28131 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:15.986522 28131 heartbeater.cc:511] Master 127.17.109.254:42793 requested a full tablet report, sending...
I20260806 20:26:15.988170 27655 ts_manager.cc:194] Registered new tserver with Master: 34e26848800a424d93565886c32903f1 (127.17.109.195:40229)
I20260806 20:26:15.989066 27655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.195:58005
I20260806 20:26:15.999097 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 20:26:16.006487 27856 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:16.004493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35588 (local address 127.17.109.254:42793)
0806 20:26:16.004675 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:16.004680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:16.005225 (+ 545us) server_negotiation.cc:407] Connection header received
0806 20:26:16.005473 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:16.005480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:16.005592 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:16.005690 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:16.006229 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35588: BlockingRecv error: recv got EOF from 127.0.0.1:35588 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 20:26:16.007037 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27715
I20260806 20:26:16.015977 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:16.008163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.17.109.193:45497)
0806 20:26:16.008785 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:16.008793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:16.008821 (+ 28us) server_negotiation.cc:407] Connection header received
0806 20:26:16.008880 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:16.008888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:16.009114 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:16.009256 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:16.009685 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:16.010843 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:16.012274 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:16.012909 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:16.013017 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:16.014301 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:16.014311 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:16.014324 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:16.014965 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:16.015017 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:16.015037 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:16.015172 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:16.015385 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 20:26:16.015545 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":301,"threads_started":1}
I20260806 20:26:16.017272 27848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:16.100063 27854 heartbeater.cc:503] Master 127.17.109.254:42793 was elected leader, sending a full tablet report...
I20260806 20:26:16.101984 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 27715
I20260806 20:26:16.121064 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27859
I20260806 20:26:16.129145 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:16.122038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59040 (local address 127.17.109.194:38093)
0806 20:26:16.122603 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:16.122610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:16.122638 (+ 28us) server_negotiation.cc:407] Connection header received
0806 20:26:16.122702 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:16.122709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:16.122946 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:16.123079 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:16.123463 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:16.124527 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:16.125870 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:16.126432 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:16.126525 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:16.127744 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:16.127755 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:16.127771 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:16.128401 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:16.128453 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:16.128473 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:16.128569 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:16.128711 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 20:26:16.128831 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":300,"threads_started":1}
I20260806 20:26:16.130465 27989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:16.219919 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 27859
I20260806 20:26:16.237916 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27997
I20260806 20:26:16.246683 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:16.239072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.17.109.195:40229)
0806 20:26:16.239786 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:16.239793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:16.239825 (+ 32us) server_negotiation.cc:407] Connection header received
0806 20:26:16.239888 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:16.239894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:16.240112 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:16.240297 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:16.240797 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:16.241877 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:16.243141 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:16.243819 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:16.243973 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:16.245143 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:16.245152 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:16.245167 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:16.245794 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:16.245844 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:16.245863 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:16.245992 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:16.246173 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 20:26:16.246322 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":350,"threads_started":1}
I20260806 20:26:16.248190 28125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:16.333673 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 27997
I20260806 20:26:16.353012 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 27621
I20260806 20:26:16.354272 27686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:16.458765 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 27621
2026-08-06T20:26:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2582 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T20:26:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:26:16Z Disabled control of system clock
I20260806 20:26:16.546247 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:39427
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:39427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 20:26:16.895069 28150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:16.895373 28150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:16.895592 28150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:16.902987 28150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:26:16.903095 28150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:16.903198 28150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:16.903301 28150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:26:16.903365 28150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:26:16.912752 28150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:39427
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:39427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:16.914017 28150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:16.915813 28150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:16.928072 28158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:16.928108 28160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:16.928520 28156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:16.928875 28150 server_base.cc:1034] running on GCE node
I20260806 20:26:16.929678 28150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:16.931056 28150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:16.932293 28150 hybrid_clock.cc:648] HybridClock initialized: now 1786047976932245 us; error 41 us; skew 500 ppm
I20260806 20:26:16.932685 28150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:26:16.935053 28150 webserver.cc:460] Webserver started at http://127.17.109.254:46517/ using document root <none> and password file <none>
I20260806 20:26:16.935842 28150 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:16.935986 28150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:16.936311 28150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:16.939250 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "e4f5cf34424849a3999547f7be9d7d4c"
format_stamp: "Formatted at 2026-08-06 20:26:16 on dist-test-slave-6k22"
server_key: "63204bdece9d8995977f1069822d9efa"
server_key_iv: "3c5710890d447320aa964ad29d6d3303"
server_key_version: "encryptionkey@0"
I20260806 20:26:16.940227 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "e4f5cf34424849a3999547f7be9d7d4c"
format_stamp: "Formatted at 2026-08-06 20:26:16 on dist-test-slave-6k22"
server_key: "63204bdece9d8995977f1069822d9efa"
server_key_iv: "3c5710890d447320aa964ad29d6d3303"
server_key_version: "encryptionkey@0"
I20260806 20:26:16.945508 28150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:26:16.949208 28166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:16.950703 28150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:26:16.950932 28150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "e4f5cf34424849a3999547f7be9d7d4c"
format_stamp: "Formatted at 2026-08-06 20:26:16 on dist-test-slave-6k22"
server_key: "63204bdece9d8995977f1069822d9efa"
server_key_iv: "3c5710890d447320aa964ad29d6d3303"
server_key_version: "encryptionkey@0"
I20260806 20:26:16.951205 28150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:16.957442 28150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:16.958312 28150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:16.958666 28150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:16.979511 28150 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:39427
I20260806 20:26:16.979532 28218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:39427 every 8 connection(s)
I20260806 20:26:16.981187 28150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:26:16.983803 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 28150
I20260806 20:26:16.984033 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:26:16.984908 17847 external_mini_cluster.cc:1428] Setting key 490a61f4e4b7a3bfbd553a43a807b4d0
I20260806 20:26:16.985847 28219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:26:16.994583 28222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:16.987192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.17.109.254:39427)
0806 20:26:16.988303 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:16.988315 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:16.988361 (+ 46us) server_negotiation.cc:407] Connection header received
0806 20:26:16.990661 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:16.990685 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:16.991153 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:16.991520 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:16.992849 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:16.992857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:16.992870 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:16.992884 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:16.992977 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:16.992998 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:16.993192 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:16.993316 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 20:26:16.993664 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":453,"threads_started":1}
I20260806 20:26:16.999333 28219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c: Bootstrap starting.
I20260806 20:26:17.002774 28219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c: Neither blocks nor log segments found. Creating new log.
I20260806 20:26:17.003947 28219 log.cc:824] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c: Log is configured to *not* fsync() on all Append() calls
I20260806 20:26:17.007153 28219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c: No bootstrap required, opened a new log
I20260806 20:26:17.014953 28219 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f5cf34424849a3999547f7be9d7d4c" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39427 } }
I20260806 20:26:17.015348 28219 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:26:17.015448 28219 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f5cf34424849a3999547f7be9d7d4c, State: Initialized, Role: FOLLOWER
I20260806 20:26:17.016098 28219 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f5cf34424849a3999547f7be9d7d4c" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39427 } }
I20260806 20:26:17.016350 28219 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:26:17.016510 28219 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:26:17.016682 28219 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:26:17.019346 28219 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f5cf34424849a3999547f7be9d7d4c" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39427 } }
I20260806 20:26:17.019960 28219 leader_election.cc:302] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4f5cf34424849a3999547f7be9d7d4c; no voters:
I20260806 20:26:17.020572 28219 leader_election.cc:288] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:26:17.020751 28224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:26:17.021539 28224 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [term 1 LEADER]: Becoming Leader. State: Replica: e4f5cf34424849a3999547f7be9d7d4c, State: Running, Role: LEADER
I20260806 20:26:17.022064 28224 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f5cf34424849a3999547f7be9d7d4c" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39427 } }
I20260806 20:26:17.022742 28219 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:26:17.025030 28226 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4f5cf34424849a3999547f7be9d7d4c. Latest consensus state: current_term: 1 leader_uuid: "e4f5cf34424849a3999547f7be9d7d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f5cf34424849a3999547f7be9d7d4c" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39427 } } }
I20260806 20:26:17.025672 28226 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [sys.catalog]: This master's current role is: LEADER
I20260806 20:26:17.025666 28225 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4f5cf34424849a3999547f7be9d7d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4f5cf34424849a3999547f7be9d7d4c" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 39427 } } }
I20260806 20:26:17.032486 28225 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c [sys.catalog]: This master's current role is: LEADER
I20260806 20:26:17.032948 28232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:26:17.038491 28232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:26:17.048236 28232 catalog_manager.cc:1347] Generated new cluster ID: db3937e96aec453c9f92685309a37b7f
I20260806 20:26:17.048390 28232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:26:17.067636 28232 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:26:17.068132 28232 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:26:17.081208 28232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4f5cf34424849a3999547f7be9d7d4c: Generated new TSK 0
I20260806 20:26:17.082139 28232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:26:17.093084 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39427
--builtin_ntp_servers=127.17.109.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 20:26:17.461843 28244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:17.462119 28244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:17.462258 28244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:17.469589 28244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:17.469691 28244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:17.469862 28244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:26:17.479660 28244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:17.480993 28244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:17.482757 28244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:17.496695 28250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:17.496703 28253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:17.496764 28251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:17.497746 28244 server_base.cc:1034] running on GCE node
I20260806 20:26:17.498345 28244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:17.499356 28244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:17.500562 28244 hybrid_clock.cc:648] HybridClock initialized: now 1786047977500561 us; error 99 us; skew 500 ppm
I20260806 20:26:17.500929 28244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:26:17.503443 28244 webserver.cc:460] Webserver started at http://127.17.109.193:42511/ using document root <none> and password file <none>
I20260806 20:26:17.504185 28244 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:17.504272 28244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:17.504650 28244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:17.507864 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "e657933da0154a58a328d3f135220779"
format_stamp: "Formatted at 2026-08-06 20:26:17 on dist-test-slave-6k22"
server_key: "d882d6c562118fd0f066dddd0c77e561"
server_key_iv: "e46001e51d5b10d7e47eaf5180c0fc9e"
server_key_version: "encryptionkey@0"
I20260806 20:26:17.508661 28244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "e657933da0154a58a328d3f135220779"
format_stamp: "Formatted at 2026-08-06 20:26:17 on dist-test-slave-6k22"
server_key: "d882d6c562118fd0f066dddd0c77e561"
server_key_iv: "e46001e51d5b10d7e47eaf5180c0fc9e"
server_key_version: "encryptionkey@0"
I20260806 20:26:17.514345 28244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260806 20:26:17.518575 28259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:17.520124 28244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:26:17.520289 28244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "e657933da0154a58a328d3f135220779"
format_stamp: "Formatted at 2026-08-06 20:26:17 on dist-test-slave-6k22"
server_key: "d882d6c562118fd0f066dddd0c77e561"
server_key_iv: "e46001e51d5b10d7e47eaf5180c0fc9e"
server_key_version: "encryptionkey@0"
I20260806 20:26:17.520521 28244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:17.526408 28244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:17.527199 28244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:17.527462 28244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:17.528393 28244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:17.529789 28244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:17.529872 28244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:17.530019 28244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:17.530118 28244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:17.564854 28244 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:38093
I20260806 20:26:17.564905 28375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:38093 every 8 connection(s)
I20260806 20:26:17.566221 28244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:26:17.567855 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 28244
I20260806 20:26:17.568056 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:26:17.569087 17847 external_mini_cluster.cc:1428] Setting key f2a8fcef483ba5fada4cf7f7265dcf4b
I20260806 20:26:17.572161 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39427
--builtin_ntp_servers=127.17.109.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 20:26:17.576943 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:17.569082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:51873 (local address 127.17.109.254:39427)
0806 20:26:17.571711 (+ 2629us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:17.571716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:17.572662 (+ 946us) server_negotiation.cc:407] Connection header received
0806 20:26:17.574428 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:17.574433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:17.574508 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:17.574633 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:17.576105 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:17.576111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:17.576122 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:17.576140 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:17.576189 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:17.576201 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:17.576366 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:17.576540 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 20:26:17.576618 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2522,"thread_start_us":2398,"threads_started":1}
I20260806 20:26:17.577955 28381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:17.569800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:39427 (local address 127.17.109.193:51873)
0806 20:26:17.572503 (+ 2703us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:17.572550 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:17.574148 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:17.574895 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:17.574903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:17.574997 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:17.575767 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:26:17.575769 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:17.575829 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:26:17.575923 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:17.576300 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:17.576312 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:17.576320 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 20:26:17.576638 (+ 318us) client_negotiation.cc:240] Negotiation successful
0806 20:26:17.576933 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":433,"threads_started":1}
I20260806 20:26:17.579823 28377 heartbeater.cc:347] Connected to a master server at 127.17.109.254:39427
I20260806 20:26:17.580183 28377 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:17.580947 28377 heartbeater.cc:511] Master 127.17.109.254:39427 requested a full tablet report, sending...
I20260806 20:26:17.583483 28184 ts_manager.cc:194] Registered new tserver with Master: e657933da0154a58a328d3f135220779 (127.17.109.193:38093)
W20260806 20:26:17.919842 28382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:17.920113 28382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:17.920279 28382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:17.927851 28382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:17.927955 28382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:17.928128 28382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:26:17.936986 28382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:17.938297 28382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:17.940088 28382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:17.953235 28390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:17.953284 28388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:17.953330 28387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:17.954192 28382 server_base.cc:1034] running on GCE node
I20260806 20:26:17.954942 28382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:17.956148 28382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:17.957406 28382 hybrid_clock.cc:648] HybridClock initialized: now 1786047977957350 us; error 64 us; skew 500 ppm
I20260806 20:26:17.957769 28382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:26:17.968461 28382 webserver.cc:460] Webserver started at http://127.17.109.194:40147/ using document root <none> and password file <none>
I20260806 20:26:17.969336 28382 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:17.969492 28382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:17.969877 28382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:17.973279 28382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "1af8ddb7d7254803b295d5da23d37f59"
format_stamp: "Formatted at 2026-08-06 20:26:17 on dist-test-slave-6k22"
server_key: "6519b4172e3b9fe5558ef07a0715e29b"
server_key_iv: "b8d32957a94c1c7d4b8494c582449742"
server_key_version: "encryptionkey@0"
I20260806 20:26:17.974175 28382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "1af8ddb7d7254803b295d5da23d37f59"
format_stamp: "Formatted at 2026-08-06 20:26:17 on dist-test-slave-6k22"
server_key: "6519b4172e3b9fe5558ef07a0715e29b"
server_key_iv: "b8d32957a94c1c7d4b8494c582449742"
server_key_version: "encryptionkey@0"
I20260806 20:26:17.980250 28382 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 20:26:17.984948 28396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:17.986636 28382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260806 20:26:17.986826 28382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "1af8ddb7d7254803b295d5da23d37f59"
format_stamp: "Formatted at 2026-08-06 20:26:17 on dist-test-slave-6k22"
server_key: "6519b4172e3b9fe5558ef07a0715e29b"
server_key_iv: "b8d32957a94c1c7d4b8494c582449742"
server_key_version: "encryptionkey@0"
I20260806 20:26:17.987071 28382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:17.995114 28382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:17.996217 28382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:17.996536 28382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:17.997843 28382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:17.999801 28382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:17.999927 28382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:18.000047 28382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:18.000118 28382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:18.037765 28382 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:33233
I20260806 20:26:18.037815 28524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:33233 every 8 connection(s)
I20260806 20:26:18.039268 28382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:26:18.043087 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 28382
I20260806 20:26:18.043321 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:26:18.043919 17847 external_mini_cluster.cc:1428] Setting key 4f339e3d0411b5cf7fa4da502d3fc8b1
I20260806 20:26:18.047222 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39427
--builtin_ntp_servers=127.17.109.212:43551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 20:26:18.048430 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:18.042147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:59109 (local address 127.17.109.254:39427)
0806 20:26:18.042388 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:18.042396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:18.044174 (+ 1778us) server_negotiation.cc:407] Connection header received
0806 20:26:18.046018 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:18.046024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:18.046092 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:18.046220 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:18.047653 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:18.047659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:18.047663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:18.047685 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:18.047723 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:18.047735 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:18.047861 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:18.048056 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 20:26:18.048130 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 20:26:18.049402 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:18.042859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:39427 (local address 127.17.109.194:59109)
0806 20:26:18.043984 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:18.044038 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:18.045734 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:18.046453 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:18.046460 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:18.046551 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:18.047258 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:26:18.047261 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:18.047299 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:26:18.047422 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:18.047833 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:18.047844 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:18.047852 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 20:26:18.048156 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 20:26:18.048422 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":363,"threads_started":1}
I20260806 20:26:18.051038 28525 heartbeater.cc:347] Connected to a master server at 127.17.109.254:39427
I20260806 20:26:18.051363 28525 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:18.052006 28525 heartbeater.cc:511] Master 127.17.109.254:39427 requested a full tablet report, sending...
I20260806 20:26:18.053540 28184 ts_manager.cc:194] Registered new tserver with Master: 1af8ddb7d7254803b295d5da23d37f59 (127.17.109.194:33233)
W20260806 20:26:18.394582 28529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:18.395006 28529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:18.395174 28529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:18.402724 28529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:18.402842 28529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:18.402953 28529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:26:18.412544 28529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:43551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:39427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:18.413898 28529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:18.415684 28529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:18.429747 28536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:18.429937 28538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:18.430356 28535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:18.430627 28529 server_base.cc:1034] running on GCE node
I20260806 20:26:18.431269 28529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:18.432335 28529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:18.433552 28529 hybrid_clock.cc:648] HybridClock initialized: now 1786047978433475 us; error 87 us; skew 500 ppm
I20260806 20:26:18.433986 28529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:26:18.436553 28529 webserver.cc:460] Webserver started at http://127.17.109.195:37387/ using document root <none> and password file <none>
I20260806 20:26:18.437258 28529 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:18.437358 28529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:18.437641 28529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:18.440758 28529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "8ab5fa634ee84c1b9c3f8c9802b014c6"
format_stamp: "Formatted at 2026-08-06 20:26:18 on dist-test-slave-6k22"
server_key: "323663111f70524150ef3ad61b24afd5"
server_key_iv: "32de1622ef57257b0bc18870a7033238"
server_key_version: "encryptionkey@0"
I20260806 20:26:18.441547 28529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "8ab5fa634ee84c1b9c3f8c9802b014c6"
format_stamp: "Formatted at 2026-08-06 20:26:18 on dist-test-slave-6k22"
server_key: "323663111f70524150ef3ad61b24afd5"
server_key_iv: "32de1622ef57257b0bc18870a7033238"
server_key_version: "encryptionkey@0"
I20260806 20:26:18.447213 28529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:26:18.450948 28545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:18.452478 28529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 20:26:18.452654 28529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "8ab5fa634ee84c1b9c3f8c9802b014c6"
format_stamp: "Formatted at 2026-08-06 20:26:18 on dist-test-slave-6k22"
server_key: "323663111f70524150ef3ad61b24afd5"
server_key_iv: "32de1622ef57257b0bc18870a7033238"
server_key_version: "encryptionkey@0"
I20260806 20:26:18.452942 28529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:18.458969 28529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:18.459900 28529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:18.460184 28529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:18.461156 28529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:18.462679 28529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:18.462769 28529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:18.462927 28529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:18.463032 28529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:18.497480 28529 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:44189
I20260806 20:26:18.497550 28672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:44189 every 8 connection(s)
I20260806 20:26:18.498939 28529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:26:18.507781 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:18.501779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:53169 (local address 127.17.109.254:39427)
0806 20:26:18.502006 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:18.502012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:18.503514 (+ 1502us) server_negotiation.cc:407] Connection header received
0806 20:26:18.505370 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:18.505376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:18.505455 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:18.505568 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:18.506959 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:18.506964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:18.506968 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:18.506984 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:18.507017 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:18.507028 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:18.507183 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:18.507369 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 20:26:18.507444 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260806 20:26:18.508373 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 28529
I20260806 20:26:18.508579 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:26:18.508843 28676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:18.502355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:39427 (local address 127.17.109.195:53169)
0806 20:26:18.503346 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:18.503391 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:18.505096 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:18.505813 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:18.505821 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:18.505913 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:18.506601 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:26:18.506604 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:18.506684 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:26:18.506774 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:18.507128 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:18.507138 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:18.507146 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 20:26:18.507457 (+ 311us) client_negotiation.cc:240] Negotiation successful
0806 20:26:18.507768 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":315,"threads_started":1}
I20260806 20:26:18.509265 17847 external_mini_cluster.cc:1428] Setting key 181c493b355a786b7ac510fc310e85ff
I20260806 20:26:18.510460 28673 heartbeater.cc:347] Connected to a master server at 127.17.109.254:39427
I20260806 20:26:18.510859 28673 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:18.511595 28673 heartbeater.cc:511] Master 127.17.109.254:39427 requested a full tablet report, sending...
I20260806 20:26:18.513104 28184 ts_manager.cc:194] Registered new tserver with Master: 8ab5fa634ee84c1b9c3f8c9802b014c6 (127.17.109.195:44189)
I20260806 20:26:18.525537 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 20:26:18.532162 28379 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:26:18.530335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.17.109.254:39427)
0806 20:26:18.530513 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:18.530517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:18.531099 (+ 582us) server_negotiation.cc:407] Connection header received
0806 20:26:18.531337 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:18.531342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:18.531416 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:18.531521 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:18.531957 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47386: BlockingRecv error: recv got EOF from 127.0.0.1:47386 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:26:18.533025 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 28244
I20260806 20:26:18.542557 28684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:18.534127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39346 (local address 127.17.109.193:38093)
0806 20:26:18.534790 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:18.534804 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:18.540130 (+ 5326us) server_negotiation.cc:407] Connection header received
0806 20:26:18.540229 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:18.540236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:18.540481 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:18.540630 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:18.541726 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:18.541734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:18.541747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:18.541767 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:18.541812 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:18.541827 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:18.541944 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:18.542079 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 20:26:18.542216 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":272,"threads_started":1}
I20260806 20:26:18.544099 28371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:18.628523 28377 heartbeater.cc:503] Master 127.17.109.254:39427 was elected leader, sending a full tablet report...
I20260806 20:26:18.630316 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 28244
I20260806 20:26:18.646380 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 28382
I20260806 20:26:18.650166 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:18.647462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.17.109.194:33233)
0806 20:26:18.648059 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:18.648066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:18.648092 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:26:18.648153 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:18.648160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:18.648405 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:18.648534 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:18.649467 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:18.649472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:18.649483 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:18.649500 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:18.649539 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:18.649553 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:18.649659 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:18.649747 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 20:26:18.649863 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":327,"threads_started":1}
I20260806 20:26:18.651528 28518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:18.740386 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 28382
I20260806 20:26:18.760551 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 28529
I20260806 20:26:18.764757 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:18.761654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.17.109.195:44189)
0806 20:26:18.762355 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:18.762362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:18.762394 (+ 32us) server_negotiation.cc:407] Connection header received
0806 20:26:18.762461 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:18.762468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:18.762735 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:18.762883 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:18.763948 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:18.763955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:18.763967 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:26:18.763985 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:18.764027 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:26:18.764041 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:18.764190 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:18.764246 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 20:26:18.764367 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":360,"threads_started":1}
I20260806 20:26:18.766050 28666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:18.850553 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 28529
I20260806 20:26:18.868805 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 28150
I20260806 20:26:18.870085 28214 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:18.974629 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 28150
2026-08-06T20:26:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2516 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:26:19 dist-test-slave-6k22 krb5kdc[28693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:26:19 dist-test-slave-6k22 krb5kdc[28693](info): set up 2 sockets
Aug 06 20:26:19 dist-test-slave-6k22 krb5kdc[28693](info): commencing operation
krb5kdc: starting...
W20260806 20:26:20.166673 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:26:20 dist-test-slave-6k22 krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:26:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:26:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:26:20.457638 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:37439
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:37439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:26:20.796836 28707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:20.797130 28707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:20.797302 28707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:20.804502 28707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:26:20.804602 28707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:20.804746 28707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:20.804831 28707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:26:20.804912 28707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:26:20.814678 28707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:37439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:37439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:20.816107 28707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:20.817471 28707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:20.829778 28713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:20.829795 28714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:20.829977 28716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:20.831247 28707 server_base.cc:1034] running on GCE node
I20260806 20:26:20.832396 28707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:20.833855 28707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:20.835110 28707 hybrid_clock.cc:648] HybridClock initialized: now 1786047980835062 us; error 60 us; skew 500 ppm
I20260806 20:26:20.835472 28707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:26:20 dist-test-slave-6k22 krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047980, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:26:20.838828 28707 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:26:20.840499 28707 webserver.cc:460] Webserver started at http://127.17.109.254:45555/ using document root <none> and password file <none>
I20260806 20:26:20.841200 28707 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:20.841359 28707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:20.841681 28707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:20.844799 28707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "7cd3a4d3c0454d548086e6310aee0d51"
format_stamp: "Formatted at 2026-08-06 20:26:20 on dist-test-slave-6k22"
server_key: "386593e5071b6514744fd501192ea00d"
server_key_iv: "bd34b07b4f0baacc4cf48307c1beed76"
server_key_version: "encryptionkey@0"
I20260806 20:26:20.845584 28707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "7cd3a4d3c0454d548086e6310aee0d51"
format_stamp: "Formatted at 2026-08-06 20:26:20 on dist-test-slave-6k22"
server_key: "386593e5071b6514744fd501192ea00d"
server_key_iv: "bd34b07b4f0baacc4cf48307c1beed76"
server_key_version: "encryptionkey@0"
I20260806 20:26:20.851024 28707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:26:20.854635 28723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:20.856112 28707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:26:20.856298 28707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "7cd3a4d3c0454d548086e6310aee0d51"
format_stamp: "Formatted at 2026-08-06 20:26:20 on dist-test-slave-6k22"
server_key: "386593e5071b6514744fd501192ea00d"
server_key_iv: "bd34b07b4f0baacc4cf48307c1beed76"
server_key_version: "encryptionkey@0"
I20260806 20:26:20.856506 28707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:20.880651 28707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:20.885365 28707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:20.885674 28707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:20.905073 28707 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:37439
I20260806 20:26:20.905107 28776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:37439 every 8 connection(s)
I20260806 20:26:20.906438 28707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:26:20.906958 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 28707
I20260806 20:26:20.907192 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:26:20.908034 17847 external_mini_cluster.cc:1428] Setting key 124fb9cf2d314f3e5e65ff2b33048a27
I20260806 20:26:20.910791 28777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 20:26:20 dist-test-slave-6k22 krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:26:20.924566 28777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51: Bootstrap starting.
I20260806 20:26:20.928323 28777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51: Neither blocks nor log segments found. Creating new log.
I20260806 20:26:20.929421 28777 log.cc:824] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51: Log is configured to *not* fsync() on all Append() calls
I20260806 20:26:20.932375 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:20.910326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.17.109.254:37439)
0806 20:26:20.911152 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:20.911170 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:20.911222 (+ 52us) server_negotiation.cc:407] Connection header received
0806 20:26:20.913605 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:20.913633 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:20.914134 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:20.914570 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:20.916259 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:20.918522 (+ 2263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:20.919912 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:20.920593 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:20.920809 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:20.924831 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:20.924839 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:20.924852 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:20.924891 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:20.928103 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:20.928910 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:20.928917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:20.928923 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:20.929084 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:20.929509 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:20.929520 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:20.929523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:20.930276 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:20.930629 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:20.930969 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 20:26:20.931320 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":359,"threads_started":1}
I20260806 20:26:20.932839 28777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51: No bootstrap required, opened a new log
I20260806 20:26:20.941517 28777 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd3a4d3c0454d548086e6310aee0d51" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37439 } }
I20260806 20:26:20.941926 28777 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:26:20.942039 28777 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd3a4d3c0454d548086e6310aee0d51, State: Initialized, Role: FOLLOWER
I20260806 20:26:20.942608 28777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd3a4d3c0454d548086e6310aee0d51" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37439 } }
I20260806 20:26:20.942863 28777 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:26:20.943018 28777 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:26:20.943199 28777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:26:20.945603 28777 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd3a4d3c0454d548086e6310aee0d51" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37439 } }
I20260806 20:26:20.946117 28777 leader_election.cc:302] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cd3a4d3c0454d548086e6310aee0d51; no voters:
I20260806 20:26:20.946780 28777 leader_election.cc:288] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:26:20.946943 28783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:26:20.947817 28783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [term 1 LEADER]: Becoming Leader. State: Replica: 7cd3a4d3c0454d548086e6310aee0d51, State: Running, Role: LEADER
I20260806 20:26:20.948433 28783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd3a4d3c0454d548086e6310aee0d51" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37439 } }
I20260806 20:26:20.949040 28777 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:26:20.951344 28785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cd3a4d3c0454d548086e6310aee0d51. Latest consensus state: current_term: 1 leader_uuid: "7cd3a4d3c0454d548086e6310aee0d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd3a4d3c0454d548086e6310aee0d51" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37439 } } }
I20260806 20:26:20.951797 28785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [sys.catalog]: This master's current role is: LEADER
I20260806 20:26:20.951669 28784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cd3a4d3c0454d548086e6310aee0d51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd3a4d3c0454d548086e6310aee0d51" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 37439 } } }
I20260806 20:26:20.951962 28784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51 [sys.catalog]: This master's current role is: LEADER
I20260806 20:26:20.956928 28791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:26:20.962713 28791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:26:20.972299 28791 catalog_manager.cc:1347] Generated new cluster ID: 99367f0310664ef2a1abe9545f5780b1
I20260806 20:26:20.972422 28791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:26:20.997377 28791 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:26:20.998888 28791 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:26:21.012797 28791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51: Generated new TSK 0
I20260806 20:26:21.013943 28791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:26:21.105801 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37439
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:26:21.472332 28804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:21.472626 28804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:21.472816 28804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:21.480682 28804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:21.480799 28804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:21.481004 28804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:26:21.491058 28804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:21.492517 28804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:21.493932 28804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:21.507203 28813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:21.507380 28810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:21.507794 28809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:21.508232 28804 server_base.cc:1034] running on GCE node
I20260806 20:26:21.508817 28804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:21.509742 28804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:21.510972 28804 hybrid_clock.cc:648] HybridClock initialized: now 1786047981510907 us; error 50 us; skew 500 ppm
I20260806 20:26:21.511263 28804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:26:21 dist-test-slave-6k22 krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047981, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:26:21.514616 28804 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:26:21.516163 28804 webserver.cc:460] Webserver started at http://127.17.109.193:37287/ using document root <none> and password file <none>
I20260806 20:26:21.516870 28804 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:21.517025 28804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:21.517333 28804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:21.520321 28804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "d81b03af27be47e28097661a59847dd3"
format_stamp: "Formatted at 2026-08-06 20:26:21 on dist-test-slave-6k22"
server_key: "48bfa16c6765a4434457f0adc1ea8bf2"
server_key_iv: "719264d96be5702b2f78371a5a0ebd40"
server_key_version: "encryptionkey@0"
I20260806 20:26:21.521111 28804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "d81b03af27be47e28097661a59847dd3"
format_stamp: "Formatted at 2026-08-06 20:26:21 on dist-test-slave-6k22"
server_key: "48bfa16c6765a4434457f0adc1ea8bf2"
server_key_iv: "719264d96be5702b2f78371a5a0ebd40"
server_key_version: "encryptionkey@0"
I20260806 20:26:21.526202 28804 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:26:21.530072 28820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:21.531736 28804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:26:21.531944 28804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "d81b03af27be47e28097661a59847dd3"
format_stamp: "Formatted at 2026-08-06 20:26:21 on dist-test-slave-6k22"
server_key: "48bfa16c6765a4434457f0adc1ea8bf2"
server_key_iv: "719264d96be5702b2f78371a5a0ebd40"
server_key_version: "encryptionkey@0"
I20260806 20:26:21.532171 28804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:21.545313 28804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:21.549681 28804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:21.549988 28804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:21.551066 28804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:21.552424 28804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:21.552495 28804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:21.552575 28804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:21.552623 28804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:21.586833 28804 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:45761
I20260806 20:26:21.586987 28939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:45761 every 8 connection(s)
I20260806 20:26:21.588213 28804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:26:21.589725 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 28804
I20260806 20:26:21.589931 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:26:21.590749 17847 external_mini_cluster.cc:1428] Setting key 62958b464d4f8e696e7dda87ebc0a1d8
Aug 06 20:26:21 dist-test-slave-6k22 krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047981, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:26:21.609712 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:21.591758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:36701 (local address 127.17.109.254:37439)
0806 20:26:21.592290 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:21.592296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:21.593269 (+ 973us) server_negotiation.cc:407] Connection header received
0806 20:26:21.595067 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:21.595073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:21.595162 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:21.595279 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:21.596886 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:21.597609 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:21.599218 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:21.599691 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:21.599827 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:21.603750 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:21.603756 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:21.603759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:21.603792 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:21.606470 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:21.607287 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:21.607293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:21.607298 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:21.607425 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:21.607929 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:21.607934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:21.607937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:21.608210 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:21.608372 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:21.609100 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 20:26:21.609339 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":276,"threads_started":1}
I20260806 20:26:21.610731 28943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:21.592360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:37439 (local address 127.17.109.193:36701)
0806 20:26:21.593091 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:21.593129 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:21.594805 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:21.595525 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:21.595533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:21.596108 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:21.596683 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:21.596700 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:21.597767 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:21.597780 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:21.599039 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:21.599052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:21.599934 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:21.599942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:21.600180 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:21.600939 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:21.600962 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:21.603466 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:21.606635 (+ 3169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:26:21.606645 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:26:21.606669 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:26:21.607150 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:26:21.607546 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:26:21.607551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:26:21.607576 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:26:21.607802 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:26:21.608344 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:21.608352 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:21.608714 (+ 362us) client_negotiation.cc:769] Sending connection context
0806 20:26:21.608985 (+ 271us) client_negotiation.cc:240] Negotiation successful
0806 20:26:21.609572 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":330,"threads_started":1}
I20260806 20:26:21.612504 28940 heartbeater.cc:347] Connected to a master server at 127.17.109.254:37439
I20260806 20:26:21.612839 28940 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:21.613580 28940 heartbeater.cc:511] Master 127.17.109.254:37439 requested a full tablet report, sending...
I20260806 20:26:21.616019 28741 ts_manager.cc:194] Registered new tserver with Master: d81b03af27be47e28097661a59847dd3 (127.17.109.193:45761)
I20260806 20:26:21.618302 28741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:36701
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:26:21.659260 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37439
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:26:21.996969 28946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:21.997304 28946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:21.997411 28946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:22.005252 28946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:22.005383 28946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:22.005501 28946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:26:22.015761 28946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:22.017095 28946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:22.018392 28946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:22.032027 28954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:22.032250 28951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:22.032255 28952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:22.032963 28946 server_base.cc:1034] running on GCE node
I20260806 20:26:22.033526 28946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:22.034441 28946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:22.035666 28946 hybrid_clock.cc:648] HybridClock initialized: now 1786047982035626 us; error 33 us; skew 500 ppm
I20260806 20:26:22.035984 28946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:26:22 dist-test-slave-6k22 krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047982, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:26:22.039203 28946 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:26:22.040632 28946 webserver.cc:460] Webserver started at http://127.17.109.194:44997/ using document root <none> and password file <none>
I20260806 20:26:22.041282 28946 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:22.041384 28946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:22.041625 28946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:22.044642 28946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "d7cdd8474c3c423b91b928541db6ff7b"
format_stamp: "Formatted at 2026-08-06 20:26:22 on dist-test-slave-6k22"
server_key: "42bdf1421de1f0cd4d015f5fcf2568ac"
server_key_iv: "729d31409422f3f936f74372e0ba9741"
server_key_version: "encryptionkey@0"
I20260806 20:26:22.045370 28946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "d7cdd8474c3c423b91b928541db6ff7b"
format_stamp: "Formatted at 2026-08-06 20:26:22 on dist-test-slave-6k22"
server_key: "42bdf1421de1f0cd4d015f5fcf2568ac"
server_key_iv: "729d31409422f3f936f74372e0ba9741"
server_key_version: "encryptionkey@0"
I20260806 20:26:22.050772 28946 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:26:22.054594 28961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:22.056288 28946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:26:22.056465 28946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "d7cdd8474c3c423b91b928541db6ff7b"
format_stamp: "Formatted at 2026-08-06 20:26:22 on dist-test-slave-6k22"
server_key: "42bdf1421de1f0cd4d015f5fcf2568ac"
server_key_iv: "729d31409422f3f936f74372e0ba9741"
server_key_version: "encryptionkey@0"
I20260806 20:26:22.056664 28946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:22.075371 28946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:22.079524 28946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:22.079916 28946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:22.080953 28946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:22.082356 28946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:22.082458 28946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:22.082619 28946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:22.082695 28946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:22.117359 28946 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:33643
I20260806 20:26:22.117398 29082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:33643 every 8 connection(s)
I20260806 20:26:22.118741 28946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:26:22.128726 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 28946
I20260806 20:26:22.128943 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:26:22.129408 17847 external_mini_cluster.cc:1428] Setting key 6897db6837cbdae7672b7575e50f4286
Aug 06 20:26:22 dist-test-slave-6k22 krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047982, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:26:22.140640 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.121411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:39303 (local address 127.17.109.254:37439)
0806 20:26:22.121967 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.121972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.122789 (+ 817us) server_negotiation.cc:407] Connection header received
0806 20:26:22.124833 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.124840 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.124942 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.125076 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:22.126637 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.127428 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.129270 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.129798 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.129953 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.133979 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:22.133985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:22.133988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:22.134022 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:22.137212 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:22.137975 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:22.137989 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:22.137992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:22.138130 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:22.138655 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:22.138661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:22.138663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:22.138952 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:22.139125 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.139942 (+ 817us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.140213 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":328,"threads_started":1}
I20260806 20:26:22.141681 29086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.121825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:37439 (local address 127.17.109.194:39303)
0806 20:26:22.122574 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:22.122615 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:22.124532 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:22.125334 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:22.125343 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:22.125871 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:22.126449 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.126467 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.127631 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.127636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.129112 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.129125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.130053 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.130061 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.130323 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.131056 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:22.131086 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:22.133705 (+ 2619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:22.137363 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:26:22.137372 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:26:22.137388 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:26:22.137832 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:26:22.138259 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:26:22.138265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:26:22.138269 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:26:22.138512 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:26:22.139096 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:22.139104 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:22.139472 (+ 368us) client_negotiation.cc:769] Sending connection context
0806 20:26:22.139839 (+ 367us) client_negotiation.cc:240] Negotiation successful
0806 20:26:22.140451 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":298,"threads_started":1}
I20260806 20:26:22.143169 29083 heartbeater.cc:347] Connected to a master server at 127.17.109.254:37439
I20260806 20:26:22.143515 29083 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:22.144549 29083 heartbeater.cc:511] Master 127.17.109.254:37439 requested a full tablet report, sending...
I20260806 20:26:22.146273 28741 ts_manager.cc:194] Registered new tserver with Master: d7cdd8474c3c423b91b928541db6ff7b (127.17.109.194:33643)
I20260806 20:26:22.147526 28741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:39303
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:26:22.198663 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37439
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:26:22.539242 29089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:26:22.539592 29089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:26:22.539745 29089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:26:22.546973 29089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:26:22.547082 29089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:26:22.547194 29089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:26:22.557519 29089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:37439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:26:22.558904 29089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:26:22.560499 29089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:26:22.577000 29094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:22.577080 29095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:26:22.577113 29097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:26:22.578320 29089 server_base.cc:1034] running on GCE node
I20260806 20:26:22.579008 29089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:26:22.580088 29089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:26:22.581298 29089 hybrid_clock.cc:648] HybridClock initialized: now 1786047982581237 us; error 60 us; skew 500 ppm
I20260806 20:26:22.581619 29089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:26:22 dist-test-slave-6k22 krb5kdc[28693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047982, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:26:22.585772 29089 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:26:22.589247 29089 webserver.cc:460] Webserver started at http://127.17.109.195:35545/ using document root <none> and password file <none>
I20260806 20:26:22.590083 29089 fs_manager.cc:359] Metadata directory not provided
I20260806 20:26:22.590204 29089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:26:22.590533 29089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:26:22.594493 29089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "30c46f03b8494d60bcb35837d19ae72f"
format_stamp: "Formatted at 2026-08-06 20:26:22 on dist-test-slave-6k22"
server_key: "1267695fbf6573c14c74523076df31a4"
server_key_iv: "a917417f78152a8f9cd01b2fa763990d"
server_key_version: "encryptionkey@0"
I20260806 20:26:22.595248 29089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "30c46f03b8494d60bcb35837d19ae72f"
format_stamp: "Formatted at 2026-08-06 20:26:22 on dist-test-slave-6k22"
server_key: "1267695fbf6573c14c74523076df31a4"
server_key_iv: "a917417f78152a8f9cd01b2fa763990d"
server_key_version: "encryptionkey@0"
I20260806 20:26:22.601635 29089 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260806 20:26:22.605530 29104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:22.607015 29089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:26:22.607192 29089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "30c46f03b8494d60bcb35837d19ae72f"
format_stamp: "Formatted at 2026-08-06 20:26:22 on dist-test-slave-6k22"
server_key: "1267695fbf6573c14c74523076df31a4"
server_key_iv: "a917417f78152a8f9cd01b2fa763990d"
server_key_version: "encryptionkey@0"
I20260806 20:26:22.607393 29089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:26:22.621734 28940 heartbeater.cc:503] Master 127.17.109.254:37439 was elected leader, sending a full tablet report...
I20260806 20:26:22.628168 29089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:26:22.632459 29089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:26:22.632763 29089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:26:22.633738 29089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:26:22.635177 29089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:26:22.635277 29089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:22.635427 29089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:26:22.635500 29089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:26:22.673017 29089 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:38499
I20260806 20:26:22.673163 29221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:38499 every 8 connection(s)
I20260806 20:26:22.674381 29089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:26:22.679102 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 29089
I20260806 20:26:22.679296 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:26:22.679878 17847 external_mini_cluster.cc:1428] Setting key 384d4375954f59eb665e781a5cf51b8e
Aug 06 20:26:22 dist-test-slave-6k22 krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047982, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:26:22.697438 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.677865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:44399 (local address 127.17.109.254:37439)
0806 20:26:22.678487 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.678496 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.679538 (+ 1042us) server_negotiation.cc:407] Connection header received
0806 20:26:22.682225 (+ 2687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.682231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.682344 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.682465 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:22.684253 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.685032 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.686658 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.687055 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.687148 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.690837 (+ 3689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:22.690842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:22.690846 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:22.690878 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:22.693646 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:22.694831 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:22.694836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:22.694838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:22.695007 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:22.695617 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:22.695623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:22.695626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:22.696050 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:22.696179 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.696960 (+ 781us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.697134 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":280,"threads_started":1}
I20260806 20:26:22.698493 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.678469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:37439 (local address 127.17.109.195:44399)
0806 20:26:22.679337 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:22.679394 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:22.681867 (+ 2473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:22.682711 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:22.682721 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:22.683405 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:26:22.684078 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.684098 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.685212 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.685217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.686524 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.686537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.687272 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.687277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.687482 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.688264 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:26:22.688295 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:26:22.690615 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:26:22.693887 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:26:22.693920 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:26:22.693938 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:26:22.694673 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:26:22.695165 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:26:22.695171 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:26:22.695174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:26:22.695444 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:26:22.696190 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:26:22.696198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:26:22.696571 (+ 373us) client_negotiation.cc:769] Sending connection context
0806 20:26:22.696839 (+ 268us) client_negotiation.cc:240] Negotiation successful
0806 20:26:22.697267 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":1831,"threads_started":1}
I20260806 20:26:22.700073 29222 heartbeater.cc:347] Connected to a master server at 127.17.109.254:37439
I20260806 20:26:22.700492 29222 heartbeater.cc:464] Registering TS with master...
I20260806 20:26:22.701352 29222 heartbeater.cc:511] Master 127.17.109.254:37439 requested a full tablet report, sending...
I20260806 20:26:22.703011 28741 ts_manager.cc:194] Registered new tserver with Master: 30c46f03b8494d60bcb35837d19ae72f (127.17.109.195:38499)
I20260806 20:26:22.704092 28741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:44399
I20260806 20:26:22.708781 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:26:22.727519 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.714358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.17.109.254:37439)
0806 20:26:22.714535 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.714539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.715077 (+ 538us) server_negotiation.cc:407] Connection header received
0806 20:26:22.715182 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.715186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.715265 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.715354 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:22.717054 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.717865 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.719382 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.719787 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.719886 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.722270 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:22.722277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:22.722281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:22.722315 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:22.724931 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:22.725652 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:22.725655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:22.725658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:22.725795 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:22.726140 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:22.726143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:22.726145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:22.726462 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:22.726640 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.727106 (+ 466us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.727277 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:26:22.731941 28741 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:26:22.739044 28741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:26:22.771862 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.762612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.17.109.193:45761)
0806 20:26:22.764458 (+ 1846us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.764469 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.764496 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:26:22.764590 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.764598 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.764933 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.765126 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:26:22.765733 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.766878 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.769572 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.770651 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.770872 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.771047 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.771314 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.771481 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1659,"thread_start_us":936,"threads_started":1}
I20260806 20:26:22.771979 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.762222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45761 (local address 127.0.0.1:38386)
0806 20:26:22.764235 (+ 2013us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:22.764262 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:22.764382 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:22.765139 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:22.765144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:22.765170 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:26:22.765531 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.765544 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.767025 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.767032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.769376 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.769392 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.770868 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.770875 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.771079 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.771131 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 20:26:22.771275 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 20:26:22.771483 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1846,"mutex_wait_us":180,"thread_start_us":1705,"threads_started":1}
I20260806 20:26:22.772553 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.753505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.17.109.194:33643)
0806 20:26:22.754098 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.754112 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.762162 (+ 8050us) server_negotiation.cc:407] Connection header received
0806 20:26:22.762445 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.762452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.762652 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.762816 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:26:22.764557 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.765943 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.769421 (+ 3478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.770876 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.771005 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.771164 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.772074 (+ 910us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.772247 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":317,"threads_started":1}
I20260806 20:26:22.772723 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.753027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47412 (local address 127.17.109.195:38499)
0806 20:26:22.762178 (+ 9151us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.762189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.762229 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:26:22.762331 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.762340 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.762611 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.762816 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:26:22.764509 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.766066 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.769476 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.770874 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.771035 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.771182 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.772216 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.772371 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8958,"thread_start_us":8783,"threads_started":1}
I20260806 20:26:22.773414 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.752923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:33643 (local address 127.0.0.1:37574)
0806 20:26:22.762037 (+ 9114us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:22.762060 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:22.762271 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:22.763142 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:22.763146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:22.763162 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:26:22.763776 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.763794 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.766078 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.766085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.769206 (+ 3121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.769222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.771433 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.771440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.771774 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.771870 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 20:26:22.772842 (+ 972us) client_negotiation.cc:240] Negotiation successful
0806 20:26:22.772980 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8988,"mutex_wait_us":319,"thread_start_us":7851,"threads_started":1}
I20260806 20:26:22.773451 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.752707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:38499 (local address 127.0.0.1:47412)
0806 20:26:22.761190 (+ 8483us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:22.761259 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:22.761477 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:22.763097 (+ 1620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:22.763104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:22.763150 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:26:22.764286 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.764300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.766200 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.766207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.769294 (+ 3087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.769308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.771435 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.771440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.771774 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.771881 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 20:26:22.772952 (+ 1071us) client_negotiation.cc:240] Negotiation successful
0806 20:26:22.773071 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8232,"mutex_wait_us":970,"thread_start_us":8917,"threads_started":1}
I20260806 20:26:22.776166 28870 tablet_service.cc:1467] Processing CreateTablet for tablet 454fbf1376d148338db78f6811c32e24 (DEFAULT_TABLE table=test-table [id=c27aa38cb7884ca2b0c2e0610d2069a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:26:22.776703 29155 tablet_service.cc:1467] Processing CreateTablet for tablet 454fbf1376d148338db78f6811c32e24 (DEFAULT_TABLE table=test-table [id=c27aa38cb7884ca2b0c2e0610d2069a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:26:22.776998 29012 tablet_service.cc:1467] Processing CreateTablet for tablet 454fbf1376d148338db78f6811c32e24 (DEFAULT_TABLE table=test-table [id=c27aa38cb7884ca2b0c2e0610d2069a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:26:22.777761 28870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 454fbf1376d148338db78f6811c32e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:26:22.777949 29155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 454fbf1376d148338db78f6811c32e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:26:22.778301 29012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 454fbf1376d148338db78f6811c32e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:26:22.789528 29241 tablet_bootstrap.cc:492] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Bootstrap starting.
I20260806 20:26:22.793306 29241 tablet_bootstrap.cc:654] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Neither blocks nor log segments found. Creating new log.
I20260806 20:26:22.794337 29241 log.cc:824] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:26:22.796271 29243 tablet_bootstrap.cc:492] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Bootstrap starting.
I20260806 20:26:22.797863 29241 tablet_bootstrap.cc:492] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: No bootstrap required, opened a new log
I20260806 20:26:22.798168 29241 ts_tablet_manager.cc:1397] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260806 20:26:22.799919 29242 tablet_bootstrap.cc:492] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Bootstrap starting.
I20260806 20:26:22.799994 29243 tablet_bootstrap.cc:654] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Neither blocks nor log segments found. Creating new log.
I20260806 20:26:22.801270 29243 log.cc:824] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Log is configured to *not* fsync() on all Append() calls
I20260806 20:26:22.803057 29242 tablet_bootstrap.cc:654] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Neither blocks nor log segments found. Creating new log.
I20260806 20:26:22.804229 29242 log.cc:824] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Log is configured to *not* fsync() on all Append() calls
I20260806 20:26:22.805462 29243 tablet_bootstrap.cc:492] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: No bootstrap required, opened a new log
I20260806 20:26:22.805737 29243 ts_tablet_manager.cc:1397] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 20:26:22.806753 29242 tablet_bootstrap.cc:492] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: No bootstrap required, opened a new log
I20260806 20:26:22.807004 29242 ts_tablet_manager.cc:1397] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260806 20:26:22.808979 29241 raft_consensus.cc:348] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.809428 29241 raft_consensus.cc:374] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:26:22.809530 29241 raft_consensus.cc:729] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c46f03b8494d60bcb35837d19ae72f, State: Initialized, Role: FOLLOWER
I20260806 20:26:22.810166 29241 consensus_queue.cc:260] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.811412 29222 heartbeater.cc:503] Master 127.17.109.254:37439 was elected leader, sending a full tablet report...
I20260806 20:26:22.812074 29241 ts_tablet_manager.cc:1428] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260806 20:26:22.815893 29242 raft_consensus.cc:348] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.816363 29242 raft_consensus.cc:374] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:26:22.816473 29242 raft_consensus.cc:729] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d81b03af27be47e28097661a59847dd3, State: Initialized, Role: FOLLOWER
I20260806 20:26:22.816839 29243 raft_consensus.cc:348] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.817258 29243 raft_consensus.cc:374] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:26:22.817379 29243 raft_consensus.cc:729] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7cdd8474c3c423b91b928541db6ff7b, State: Initialized, Role: FOLLOWER
I20260806 20:26:22.817205 29242 consensus_queue.cc:260] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.818949 29242 ts_tablet_manager.cc:1428] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260806 20:26:22.817967 29243 consensus_queue.cc:260] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.820102 29083 heartbeater.cc:503] Master 127.17.109.254:37439 was elected leader, sending a full tablet report...
I20260806 20:26:22.820715 29243 ts_tablet_manager.cc:1428] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260806 20:26:22.840735 28941 tablet.cc:2369] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:26:22.859980 29247 raft_consensus.cc:482] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:26:22.860373 29247 raft_consensus.cc:504] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.862259 29247 leader_election.cc:288] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7cdd8474c3c423b91b928541db6ff7b (127.17.109.194:33643), d81b03af27be47e28097661a59847dd3 (127.17.109.193:45761)
I20260806 20:26:22.868706 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.862732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:41641 (local address 127.17.109.193:45761)
0806 20:26:22.862944 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.862952 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.863193 (+ 241us) server_negotiation.cc:407] Connection header received
0806 20:26:22.863472 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.863476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.863551 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.863726 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:26:22.864172 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.864937 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.866734 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.867692 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.867872 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.867955 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.868258 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.868400 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:26:22.868798 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.862666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45761 (local address 127.17.109.195:41641)
0806 20:26:22.863080 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:22.863102 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:22.863277 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:22.863743 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:22.863748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:22.863776 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:26:22.864018 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.864027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.865092 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.865099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.866551 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.866563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.867859 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.867866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.868073 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.868099 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 20:26:22.868233 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 20:26:22.868406 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":3,"spinlock_wait_cycles":24832}
I20260806 20:26:22.868835 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.862879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:33115 (local address 127.17.109.194:33643)
0806 20:26:22.863103 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.863110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.863763 (+ 653us) server_negotiation.cc:407] Connection header received
0806 20:26:22.863955 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.863960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.864029 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.864126 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:26:22.864657 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.865348 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.867011 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.867870 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.868057 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.868127 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.868440 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.868569 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:26:22.868983 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.862770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:33643 (local address 127.17.109.195:33115)
0806 20:26:22.863645 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 20:26:22.863662 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:26:22.863818 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:26:22.864138 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:26:22.864142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:26:22.864164 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:26:22.864494 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.864502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.865496 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.865501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.866850 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:26:22.866861 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.868072 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.868078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:26:22.868251 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.868307 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 20:26:22.868412 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 20:26:22.868548 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":22,"thread_start_us":465,"threads_started":1}
I20260806 20:26:22.869783 28891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "454fbf1376d148338db78f6811c32e24" candidate_uuid: "30c46f03b8494d60bcb35837d19ae72f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d81b03af27be47e28097661a59847dd3" is_pre_election: true
I20260806 20:26:22.870390 28891 raft_consensus.cc:2454] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30c46f03b8494d60bcb35837d19ae72f in term 0.
I20260806 20:26:22.870311 29034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "454fbf1376d148338db78f6811c32e24" candidate_uuid: "30c46f03b8494d60bcb35837d19ae72f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7cdd8474c3c423b91b928541db6ff7b" is_pre_election: true
I20260806 20:26:22.870836 29034 raft_consensus.cc:2454] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30c46f03b8494d60bcb35837d19ae72f in term 0.
W20260806 20:26:22.871070 29084 tablet.cc:2369] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:26:22.871157 29106 leader_election.cc:302] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30c46f03b8494d60bcb35837d19ae72f, d81b03af27be47e28097661a59847dd3; no voters:
I20260806 20:26:22.871615 29247 raft_consensus.cc:2781] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:26:22.871786 29247 raft_consensus.cc:482] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:26:22.871897 29247 raft_consensus.cc:3037] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:26:22.874075 29247 raft_consensus.cc:504] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.874679 29247 leader_election.cc:288] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [CANDIDATE]: Term 1 election: Requested vote from peers d7cdd8474c3c423b91b928541db6ff7b (127.17.109.194:33643), d81b03af27be47e28097661a59847dd3 (127.17.109.193:45761)
I20260806 20:26:22.875216 29034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "454fbf1376d148338db78f6811c32e24" candidate_uuid: "30c46f03b8494d60bcb35837d19ae72f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7cdd8474c3c423b91b928541db6ff7b"
I20260806 20:26:22.875201 28891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "454fbf1376d148338db78f6811c32e24" candidate_uuid: "30c46f03b8494d60bcb35837d19ae72f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d81b03af27be47e28097661a59847dd3"
I20260806 20:26:22.875435 29034 raft_consensus.cc:3037] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:26:22.875445 28891 raft_consensus.cc:3037] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:26:22.878386 29034 raft_consensus.cc:2454] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30c46f03b8494d60bcb35837d19ae72f in term 1.
I20260806 20:26:22.878846 29107 leader_election.cc:302] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30c46f03b8494d60bcb35837d19ae72f, d7cdd8474c3c423b91b928541db6ff7b; no voters:
I20260806 20:26:22.878923 28891 raft_consensus.cc:2454] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30c46f03b8494d60bcb35837d19ae72f in term 1.
I20260806 20:26:22.879204 29247 raft_consensus.cc:2781] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:26:22.879781 29247 raft_consensus.cc:686] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 1 LEADER]: Becoming Leader. State: Replica: 30c46f03b8494d60bcb35837d19ae72f, State: Running, Role: LEADER
I20260806 20:26:22.880266 29247 consensus_queue.cc:237] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } }
I20260806 20:26:22.884806 28741 catalog_manager.cc:5495] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f reported cstate change: term changed from 0 to 1, leader changed from <none> to 30c46f03b8494d60bcb35837d19ae72f (127.17.109.195). New cstate: current_term: 1 leader_uuid: "30c46f03b8494d60bcb35837d19ae72f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30c46f03b8494d60bcb35837d19ae72f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 38499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 } health_report { overall_health: UNKNOWN } } }
I20260806 20:26:22.922645 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.915936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.17.109.195:38499)
0806 20:26:22.916136 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.916143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.916277 (+ 134us) server_negotiation.cc:407] Connection header received
0806 20:26:22.916561 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.916566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.916650 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.916825 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:26:22.917231 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.918258 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.920252 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.920718 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.920877 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:22.921451 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:26:22.921975 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:26:22.922172 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:22.922261 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 20:26:22.922362 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 20:26:22.926110 29223 tablet.cc:2369] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:26:22.930980 29034 raft_consensus.cc:1261] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 1 FOLLOWER]: Refusing update from remote peer 30c46f03b8494d60bcb35837d19ae72f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:26:22.930974 28891 raft_consensus.cc:1261] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 1 FOLLOWER]: Refusing update from remote peer 30c46f03b8494d60bcb35837d19ae72f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:26:22.931968 29251 consensus_queue.cc:1035] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7cdd8474c3c423b91b928541db6ff7b" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 33643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:26:22.932353 29247 consensus_queue.cc:1035] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d81b03af27be47e28097661a59847dd3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:26:22.943181 29256 mvcc.cc:205] Tried to move back new op lower bound from 7315652538075672576 to 7315652537881407488. Current Snapshot: MvccSnapshot[applied={T|T < 7315652538075672576}]
I20260806 20:26:22.944937 29255 mvcc.cc:205] Tried to move back new op lower bound from 7315652538075672576 to 7315652537881407488. Current Snapshot: MvccSnapshot[applied={T|T < 7315652538075672576}]
I20260806 20:26:22.961882 28741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:26:22.962189 28741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:26:22.966831 28741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cd3a4d3c0454d548086e6310aee0d51: Sending DeleteTablet for 3 replicas of tablet 454fbf1376d148338db78f6811c32e24
I20260806 20:26:22.968134 28870 tablet_service.cc:1514] Processing DeleteTablet for tablet 454fbf1376d148338db78f6811c32e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:26:22 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:38386
I20260806 20:26:22.968372 29155 tablet_service.cc:1514] Processing DeleteTablet for tablet 454fbf1376d148338db78f6811c32e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:26:22 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:47412
I20260806 20:26:22.968405 28869 tablet_service.cc:1514] Processing DeleteTablet for tablet 454fbf1376d148338db78f6811c32e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:26:22 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:38386
I20260806 20:26:22.969229 29265 tablet_replica.cc:330] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: stopping tablet replica
I20260806 20:26:22.969193 29012 tablet_service.cc:1514] Processing DeleteTablet for tablet 454fbf1376d148338db78f6811c32e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:26:22 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:37574
I20260806 20:26:22.969807 29265 raft_consensus.cc:2229] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:26:22.970120 29267 tablet_replica.cc:330] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: stopping tablet replica
I20260806 20:26:22.970484 29265 raft_consensus.cc:2258] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:26:22.970669 29267 raft_consensus.cc:2229] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:26:22.971436 29267 raft_consensus.cc:2258] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:26:22.971519 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 28804
I20260806 20:26:22.973290 29265 ts_tablet_manager.cc:1905] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:26:22.973941 29267 ts_tablet_manager.cc:1905] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:26:22.981333 29265 ts_tablet_manager.cc:1918] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:26:22.981581 29265 log.cc:1196] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/454fbf1376d148338db78f6811c32e24
I20260806 20:26:22.982333 29265 ts_tablet_manager.cc:1939] T 454fbf1376d148338db78f6811c32e24 P d81b03af27be47e28097661a59847dd3: Deleting consensus metadata
I20260806 20:26:22.983362 29267 ts_tablet_manager.cc:1918] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:26:22.983644 29267 log.cc:1196] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/454fbf1376d148338db78f6811c32e24
I20260806 20:26:22.984308 29267 ts_tablet_manager.cc:1939] T 454fbf1376d148338db78f6811c32e24 P d7cdd8474c3c423b91b928541db6ff7b: Deleting consensus metadata
I20260806 20:26:22.984820 28725 catalog_manager.cc:4841] TS d81b03af27be47e28097661a59847dd3 (127.17.109.193:45761): tablet 454fbf1376d148338db78f6811c32e24 (table test-table [id=c27aa38cb7884ca2b0c2e0610d2069a9]) successfully deleted
W20260806 20:26:22.985566 28725 catalog_manager.cc:4808] TS d81b03af27be47e28097661a59847dd3 (127.17.109.193:45761): delete failed for tablet 454fbf1376d148338db78f6811c32e24 because the tablet was not found. No further retry: Not found: Tablet not found: 454fbf1376d148338db78f6811c32e24
I20260806 20:26:22.986332 28726 catalog_manager.cc:4841] TS d7cdd8474c3c423b91b928541db6ff7b (127.17.109.194:33643): tablet 454fbf1376d148338db78f6811c32e24 (table test-table [id=c27aa38cb7884ca2b0c2e0610d2069a9]) successfully deleted
I20260806 20:26:22.988731 29266 tablet_replica.cc:330] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: stopping tablet replica
I20260806 20:26:22.989219 29266 raft_consensus.cc:2229] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:26:22.989928 29266 raft_consensus.cc:2258] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:26:22.992213 29266 ts_tablet_manager.cc:1905] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:26:22.997496 29266 ts_tablet_manager.cc:1918] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:26:22.997685 29266 log.cc:1196] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/454fbf1376d148338db78f6811c32e24
I20260806 20:26:22.998318 29266 ts_tablet_manager.cc:1939] T 454fbf1376d148338db78f6811c32e24 P 30c46f03b8494d60bcb35837d19ae72f: Deleting consensus metadata
I20260806 20:26:23.000437 28725 catalog_manager.cc:4841] TS 30c46f03b8494d60bcb35837d19ae72f (127.17.109.195:38499): tablet 454fbf1376d148338db78f6811c32e24 (table test-table [id=c27aa38cb7884ca2b0c2e0610d2069a9]) successfully deleted
Aug 06 20:26:22 dist-test-slave-6k22 krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:26:23.009478 29240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:22.974265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.17.109.193:45761)
0806 20:26:22.974433 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:22.974441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:22.983185 (+ 8744us) server_negotiation.cc:407] Connection header received
0806 20:26:22.983411 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:22.983417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:22.983512 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:22.983675 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:22.986447 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.987634 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.989690 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:22.990376 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:22.990525 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:23.002155 (+ 11630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:23.002180 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:23.002205 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:23.002273 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:23.006318 (+ 4045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:23.007055 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:23.007067 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:23.007071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:23.007260 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:23.007690 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:23.007696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:23.007700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:23.008228 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:23.008547 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:23.008762 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 20:26:23.009022 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 20:26:23.010221 28935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:23.125279 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 28804
I20260806 20:26:23.145627 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 28946
Aug 06 20:26:23 dist-test-slave-6k22 krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:26:23.163396 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:23.146467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37582 (local address 127.17.109.194:33643)
0806 20:26:23.146667 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:23.146674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:23.146810 (+ 136us) server_negotiation.cc:407] Connection header received
0806 20:26:23.147048 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:23.147054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:23.147149 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:23.147272 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:23.148717 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:23.149695 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:23.151153 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:23.151800 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:23.151976 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:23.156624 (+ 4648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:23.156642 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:23.156661 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:23.156711 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:23.159916 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:23.160796 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:23.160803 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:23.160806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:23.160941 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:23.161426 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:23.161433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:23.161437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:23.161967 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:23.162287 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:23.162649 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 20:26:23.162887 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 20:26:23.164166 29077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:23.254913 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 28946
I20260806 20:26:23.274340 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 29089
Aug 06 20:26:23 dist-test-slave-6k22 krb5kdc[28693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:26:23.292939 29237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:26:23.275259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47436 (local address 127.17.109.195:38499)
0806 20:26:23.275427 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 20:26:23.275434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:26:23.275518 (+ 84us) server_negotiation.cc:407] Connection header received
0806 20:26:23.275803 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:26:23.275807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:26:23.275874 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:26:23.275994 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:26:23.277332 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:23.278221 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:23.279477 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:26:23.280247 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:26:23.280395 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:26:23.284988 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:26:23.285002 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:26:23.285023 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:26:23.285090 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:26:23.289609 (+ 4519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:23.290333 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:23.290343 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:23.290347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:23.290560 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:26:23.290948 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:26:23.290957 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:26:23.290963 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:26:23.291489 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:26:23.291895 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0806 20:26:23.292206 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 20:26:23.292470 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:26:23.293853 29217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:23.416709 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 29089
I20260806 20:26:23.438861 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 28707
I20260806 20:26:23.440127 28772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:26:23.553565 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 28707
2026-08-06T20:26:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4579 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:26:23 dist-test-slave-6k22 krb5kdc[29275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:26:23 dist-test-slave-6k22 krb5kdc[29275](info): set up 2 sockets
Aug 06 20:26:23 dist-test-slave-6k22 krb5kdc[29275](info): commencing operation
krb5kdc: starting...
W20260806 20:26:24.744463 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:26:24 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786047984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:26:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:26:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.109.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.109.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/rangeradmin_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/rangeradmin_127.17.109.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.109.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.109.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/rangerlookup_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/rangerlookup_127.17.109.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.109.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.109.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/HTTP_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/HTTP_127.17.109.212@KRBTEST.COM.keytab.
I20260806 20:26:25.162400 17847 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskcvsjCP/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/postgres -l logfile start
2026-08-06 20:26:27.905 UTC [29306] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 20:26:27.905 UTC [29306] LOG: listening on IPv4 address "127.17.109.212", port 49073
2026-08-06 20:26:27.909 UTC [29306] LOG: listening on Unix socket "/tmp/.s.PGSQL.49073"
2026-08-06 20:26:27.934 UTC [29311] LOG: database system was shut down at 2026-08-06 20:26:26 UTC
2026-08-06 20:26:27.938 UTC [29306] LOG: database system is ready to accept connections
I20260806 20:26:29.037249 17847 mini_postgres.cc:96] Postgres bound to 49073
2026-08-06 20:26:29.064 UTC [29319] FATAL: database "slave" does not exist
127.17.109.212:49073 - accepting connections
I20260806 20:26:29.065752 17847 mini_ranger.cc:161] Starting Ranger...
I20260806 20:26:29.101806 17847 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 20:26:29.623800 17847 mini_ranger.cc:87] Created ranger Postgres database
I20260806 20:26:29.623991 17847 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-admin
2026-08-06 20:26:30,034 [I] DB FLAVOR :POSTGRES
2026-08-06 20:26:30,035 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 20:26:30,035 [I] Checking connection..
2026-08-06 20:26:30,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 20:26:30,332 [I] Checking connection passed.
2026-08-06 20:26:30,332 [I] --------- Verifying version history table ---------
2026-08-06 20:26:30,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 20:26:30,606 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 20:26:30,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 20:26:30,874 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 20:26:30,874 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 20:26:30,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 20:26:31.119 UTC [29389] WARNING: there is no transaction in progress
2026-08-06 20:26:31,126 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 20:26:31,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 20:26:31,405 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 20:26:31,405 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 20:26:31,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 20:26:31,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 20:26:31,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-06 20:26:32,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 20:26:32,456 [I] Table x_portal_user does not exist in database ranger
2026-08-06 20:26:32,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 20:26:32,705 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 20:26:32,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 20:26:32,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 20:26:33,239 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 20:26:33,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 20:26:34.264 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.281 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.297 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.313 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.334 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.354 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.412 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.419 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.433 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.444 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.454 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.470 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.482 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.491 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.499 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:34.506 UTC [29505] WARNING: there is no transaction in progress
2026-08-06 20:26:35,214 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 20:26:35,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 20:26:35,472 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 20:26:35,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 20:26:35,730 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 20:26:35,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 20:26:36,011 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 20:26:36,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 20:26:36,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 20:26:36,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-06 20:26:36,747 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 20:26:36,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 20:26:37,000 [I] DB_PATCHES have already been applied
I20260806 20:26:37.006227 17847 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-admin:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskcvsjCP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 20:26:37.006419 17847 mini_ranger.cc:193] Using host: 127.17.109.212
I20260806 20:26:37.028777 17847 mini_ranger.cc:227] Ranger admin URL: http://127.17.109.212:44035
Aug 06, 2026 8:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 8:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 8:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 8:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 8:26:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 8:26:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 8:26:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 20:26:38.325331 17847 mini_ranger.cc:160] Time spent starting Ranger: real 9.260s user 0.000s sys 0.069s
Aug 06, 2026 8:26:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 8:26:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 8:26:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 8:26:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 8:27:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 20:27:01.243280 17847 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.109.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.109.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/rangerkms_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/rangerkms_127.17.109.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.109.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.109.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.109.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/HTTP_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.109.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/HTTP_127.17.109.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 20:27:01 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048021, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 20:27:01.461143 17847 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 20:27:01.488756 17847 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 20:27:02.407217 17847 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 20:27:02.407369 17847 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms
I20260806 20:27:02.407541 17847 mini_ranger_kms.cc:227] Using postgres at 127.17.109.212:49073
2026-08-06 20:27:02,909 [I] DB FLAVOR :POSTGRES
2026-08-06 20:27:02,909 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 20:27:02,909 [I] Checking connection
2026-08-06 20:27:02,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 20:27:03,166 [I] connection success
2026-08-06 20:27:03,167 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 20:27:03,167 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 20:27:03,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 20:27:03,405 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 20:27:03,405 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 20:27:03,405 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 20:27:03,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:49073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 20:27:03,693 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 20:27:04.244817 17847 mini_ranger_kms.cc:310] Created kms service
I20260806 20:27:04.490440 17847 mini_ranger_kms.cc:326] Created kudu user
I20260806 20:27:04.536109 17847 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 20:27:04 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048021, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:27:04.967916 17847 mini_ranger_kms.cc:384] Added ranger policy
I20260806 20:27:04.969383 17847 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms:/tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 20:27:04.969468 17847 mini_ranger_kms.cc:240] Using host: 127.17.109.212
I20260806 20:27:05.003460 17847 mini_ranger_kms.cc:276] Ranger KMS PID: 29696
I20260806 20:27:05.003755 17847 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.109.212:33117
Aug 06, 2026 8:27:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 8:27:05 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 8:27:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 8:27:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 8:27:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 8:27:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 20:27:07 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786048027, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.109.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:27:07.494166 17847 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.033s user 0.148s sys 0.401s
I20260806 20:27:07.494586 17847 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 20:27:07.494839 17847 mini_ranger_kms.cc:401] 127.17.109.212:33117/kms/v1/keys
Aug 06 20:27:07 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048027, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.109.212@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
Aug 06 20:27:08 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048027, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.109.212@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
Aug 06 20:27:10 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:27:10.761408 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:41967
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:41967
--ranger_config_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33117/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:27:11.132750 29749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:27:11.133131 29749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:27:11.133251 29749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:27:11.140368 29749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:27:11.140475 29749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:27:11.140522 29749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:27:11.140570 29749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:27:11.140619 29749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:27:11.150701 29749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.109.212:33117/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:41967
--ranger_config_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:41967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:27:11.151962 29749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:27:11.153198 29749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:27:11.166723 29754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:27:11.166854 29755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:27:11.166823 29757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:27:11.167915 29749 server_base.cc:1034] running on GCE node
I20260806 20:27:11.168772 29749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:27:11.170264 29749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:27:11.171521 29749 hybrid_clock.cc:648] HybridClock initialized: now 1786048031171466 us; error 42 us; skew 500 ppm
I20260806 20:27:11.171958 29749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:27:11 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048031, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:27:11.180258 29749 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:27:11.181785 29749 webserver.cc:460] Webserver started at http://127.17.109.254:44077/ using document root <none> and password file <none>
I20260806 20:27:11.182435 29749 fs_manager.cc:359] Metadata directory not provided
I20260806 20:27:11.182585 29749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:27:11.182894 29749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 20:27:11 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048031, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:27:11.242451 29749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "f938b9d9781341eda5374f564c0fd67d"
format_stamp: "Formatted at 2026-08-06 20:27:11 on dist-test-slave-6k22"
server_key: "ef57618592ec6dba800e47960834ffdc"
server_key_iv: "a723e7196eda3b8e1d6847b45d606a88"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:11.243355 29749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "f938b9d9781341eda5374f564c0fd67d"
format_stamp: "Formatted at 2026-08-06 20:27:11 on dist-test-slave-6k22"
server_key: "ef57618592ec6dba800e47960834ffdc"
server_key_iv: "a723e7196eda3b8e1d6847b45d606a88"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:11.249133 29749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 20:27:11.270130 29766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:11.271845 29749 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 20:27:11.272069 29749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "f938b9d9781341eda5374f564c0fd67d"
format_stamp: "Formatted at 2026-08-06 20:27:11 on dist-test-slave-6k22"
server_key: "ef57618592ec6dba800e47960834ffdc"
server_key_iv: "a723e7196eda3b8e1d6847b45d606a88"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:11.272359 29749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:27:11.288368 29749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:27:11.293298 29749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:27:11.293623 29749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:27:11.316345 29749 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:41967
I20260806 20:27:11.316399 29825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:41967 every 8 connection(s)
I20260806 20:27:11.318014 29749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:27:11.322844 29826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:27:11.327001 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 29749
I20260806 20:27:11.327248 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:27:11.327991 17847 external_mini_cluster.cc:1428] Setting key c57d4bafb8c64790aa246dbc221ed5f6
I20260806 20:27:11.337332 29826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d: Bootstrap starting.
I20260806 20:27:11.341373 29826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d: Neither blocks nor log segments found. Creating new log.
I20260806 20:27:11.342799 29826 log.cc:824] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d: Log is configured to *not* fsync() on all Append() calls
I20260806 20:27:11.347218 29826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d: No bootstrap required, opened a new log
Aug 06 20:27:11 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:27:11.358388 29826 raft_consensus.cc:348] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f938b9d9781341eda5374f564c0fd67d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41967 } }
I20260806 20:27:11.358870 29826 raft_consensus.cc:374] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:27:11.358970 29826 raft_consensus.cc:729] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f938b9d9781341eda5374f564c0fd67d, State: Initialized, Role: FOLLOWER
I20260806 20:27:11.359862 29826 consensus_queue.cc:260] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f938b9d9781341eda5374f564c0fd67d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41967 } }
I20260806 20:27:11.360147 29826 raft_consensus.cc:388] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:27:11.360289 29826 raft_consensus.cc:482] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:27:11.360455 29826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:27:11.363809 29826 raft_consensus.cc:504] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f938b9d9781341eda5374f564c0fd67d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41967 } }
I20260806 20:27:11.364480 29826 leader_election.cc:302] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f938b9d9781341eda5374f564c0fd67d; no voters:
I20260806 20:27:11.365191 29826 leader_election.cc:288] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:27:11.365372 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:11.330253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54514 (local address 127.17.109.254:41967)
0806 20:27:11.344268 (+ 14015us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:11.344290 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:11.344355 (+ 65us) server_negotiation.cc:407] Connection header received
0806 20:27:11.346472 (+ 2117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:11.346497 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:11.347082 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:11.347601 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:27:11.349087 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:11.350512 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:11.351837 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:11.352516 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:11.352722 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:11.356598 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:27:11.356606 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:27:11.356621 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:27:11.356655 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:27:11.360882 (+ 4227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:11.361593 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:11.361601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:11.361610 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:11.361778 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:11.362213 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:11.362218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:11.362221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:11.362762 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:27:11.363075 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:11.363510 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 20:27:11.363987 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13655,"thread_start_us":13349,"threads_started":1}
I20260806 20:27:11.365371 29831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:27:11.369872 29831 raft_consensus.cc:686] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [term 1 LEADER]: Becoming Leader. State: Replica: f938b9d9781341eda5374f564c0fd67d, State: Running, Role: LEADER
I20260806 20:27:11.370406 29831 consensus_queue.cc:237] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f938b9d9781341eda5374f564c0fd67d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41967 } }
I20260806 20:27:11.373742 29826 sys_catalog.cc:564] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:27:11.376448 29833 sys_catalog.cc:455] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f938b9d9781341eda5374f564c0fd67d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f938b9d9781341eda5374f564c0fd67d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41967 } } }
I20260806 20:27:11.376511 29832 sys_catalog.cc:455] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f938b9d9781341eda5374f564c0fd67d. Latest consensus state: current_term: 1 leader_uuid: "f938b9d9781341eda5374f564c0fd67d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f938b9d9781341eda5374f564c0fd67d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41967 } } }
I20260806 20:27:11.376998 29833 sys_catalog.cc:458] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [sys.catalog]: This master's current role is: LEADER
I20260806 20:27:11.377051 29832 sys_catalog.cc:458] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d [sys.catalog]: This master's current role is: LEADER
I20260806 20:27:11.386133 29837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:27:11.393478 29837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:27:11.414533 29837 catalog_manager.cc:1347] Generated new cluster ID: 281e775ad8a049499416df5f7cbbebee
I20260806 20:27:11.414664 29837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:27:11.434311 29837 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:27:11.436395 29837 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:27:11.446574 29837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d: Generated new TSK 0
I20260806 20:27:11.447911 29837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:27:11.453096 29826 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:27:12.316011 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33117/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41967
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:27:12 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786048032, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 20:27:12.756628 29873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:27:12.756963 29873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:27:12.757141 29873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:27:12.765051 29873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:27:12.765218 29873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:27:12.765417 29873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:27:12.778009 29873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.109.212:33117/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:27:12.779986 29873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:27:12.781832 29873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:27:12.802769 29884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:27:12.803241 29883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:27:12.810740 29887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:27:12.812574 29873 server_base.cc:1034] running on GCE node
I20260806 20:27:12.813313 29873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:27:12.815228 29873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:27:12.816613 29873 hybrid_clock.cc:648] HybridClock initialized: now 1786048032816516 us; error 138 us; skew 500 ppm
I20260806 20:27:12.816984 29873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:27:12 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048032, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:27:12.820600 29873 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:27:12.822655 29873 webserver.cc:460] Webserver started at http://127.17.109.193:34605/ using document root <none> and password file <none>
I20260806 20:27:12.823612 29873 fs_manager.cc:359] Metadata directory not provided
I20260806 20:27:12.823765 29873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:27:12.824153 29873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 20:27:12 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048032, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:27:12.843950 29873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "173b2d3ab81046c6982c1323f8ccb7f5"
format_stamp: "Formatted at 2026-08-06 20:27:12 on dist-test-slave-6k22"
server_key: "454585b2447cc6de28a6a1411a024f89"
server_key_iv: "db89e842fc7178efc6acd965454cae04"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:12.845252 29873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "173b2d3ab81046c6982c1323f8ccb7f5"
format_stamp: "Formatted at 2026-08-06 20:27:12 on dist-test-slave-6k22"
server_key: "454585b2447cc6de28a6a1411a024f89"
server_key_iv: "db89e842fc7178efc6acd965454cae04"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:12.852641 29873 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260806 20:27:12.892335 29894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:12.894476 29873 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.014s sys 0.005s
I20260806 20:27:12.894691 29873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "173b2d3ab81046c6982c1323f8ccb7f5"
format_stamp: "Formatted at 2026-08-06 20:27:12 on dist-test-slave-6k22"
server_key: "454585b2447cc6de28a6a1411a024f89"
server_key_iv: "db89e842fc7178efc6acd965454cae04"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:12.894963 29873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:27:12.956625 29873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:27:12.962091 29873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:27:12.962502 29873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:27:12.984079 29873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:27:12.991816 29873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:27:12.992035 29873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:12.993115 29873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:27:12.993260 29873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:27:13 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048032, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:27:13.180769 29873 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:40163
I20260806 20:27:13.180804 30014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:40163 every 8 connection(s)
I20260806 20:27:13.182327 29873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:27:13.183343 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 29873
I20260806 20:27:13.183738 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:27:13.184818 17847 external_mini_cluster.cc:1428] Setting key 6f6faf986e56ecf4028c8b6b302865a3
Aug 06 20:27:13 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048032, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
I20260806 20:27:13.228463 30017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:13.192915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:54595 (local address 127.17.109.254:41967)
0806 20:27:13.208046 (+ 15131us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:13.208055 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:13.208087 (+ 32us) server_negotiation.cc:407] Connection header received
0806 20:27:13.208176 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:13.208185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:13.208312 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:13.209234 (+ 922us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:27:13.211100 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:13.212242 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:13.214694 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:13.215329 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:13.216436 (+ 1107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:13.220315 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:27:13.220321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:27:13.220326 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:27:13.220365 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:27:13.224002 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:13.224991 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:13.224996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:13.225000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:13.225137 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:13.225653 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:13.225657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:13.225661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:13.226008 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:27:13.227601 (+ 1593us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:13.227766 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 20:27:13.228034 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14934,"thread_start_us":14809,"threads_started":1}
I20260806 20:27:13.229319 30016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:13.192204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:41967 (local address 127.17.109.193:54595)
0806 20:27:13.199288 (+ 7084us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:13.199351 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 20:27:13.202660 (+ 3309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:27:13.209519 (+ 6859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:27:13.209529 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:27:13.210015 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:27:13.210844 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:13.210873 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:13.212440 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:13.212445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:13.214483 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:13.214499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:13.215627 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:13.215637 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:13.215953 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:13.216926 (+ 973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:27:13.216957 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:27:13.220007 (+ 3050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:27:13.224174 (+ 4167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:27:13.224184 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:27:13.224202 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:27:13.224812 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:27:13.225272 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:27:13.225276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:27:13.225280 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:27:13.225509 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:27:13.226156 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:27:13.226163 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:27:13.226624 (+ 461us) client_negotiation.cc:769] Sending connection context
0806 20:27:13.226928 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 20:27:13.227508 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6759,"thread_start_us":6528,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
I20260806 20:27:13.231353 30015 heartbeater.cc:347] Connected to a master server at 127.17.109.254:41967
I20260806 20:27:13.231750 30015 heartbeater.cc:464] Registering TS with master...
I20260806 20:27:13.232756 30015 heartbeater.cc:511] Master 127.17.109.254:41967 requested a full tablet report, sending...
I20260806 20:27:13.235801 29783 ts_manager.cc:194] Registered new tserver with Master: 173b2d3ab81046c6982c1323f8ccb7f5 (127.17.109.193:40163)
I20260806 20:27:13.238173 29783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:54595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:27:13.275946 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33117/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41967
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:27:13 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048032, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
W20260806 20:27:13.763124 30022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:27:13.763469 30022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:27:13.763684 30022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:27:13.771215 30022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:27:13.771376 30022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:27:13.771589 30022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:27:13.782985 30022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.109.212:33117/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:27:13.784612 30022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:27:13.786072 30022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:27:13.802361 30037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:27:13.803020 30036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:27:13.803439 30039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:27:13.803889 30022 server_base.cc:1034] running on GCE node
I20260806 20:27:13.804533 30022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:27:13.805599 30022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:27:13.806869 30022 hybrid_clock.cc:648] HybridClock initialized: now 1786048033806769 us; error 79 us; skew 500 ppm
I20260806 20:27:13.807256 30022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:27:13 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:27:13.810782 30022 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:27:13.812490 30022 webserver.cc:460] Webserver started at http://127.17.109.194:39719/ using document root <none> and password file <none>
I20260806 20:27:13.813169 30022 fs_manager.cc:359] Metadata directory not provided
I20260806 20:27:13.813314 30022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:27:13.813628 30022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 20:27:13 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:27:13.831076 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "01eeb4c96b1f4097ac1ef4dd96d57660"
format_stamp: "Formatted at 2026-08-06 20:27:13 on dist-test-slave-6k22"
server_key: "70591a849977b1c65438e7a280695dae"
server_key_iv: "ac1cae003d2c47f00f46e8fb60063b07"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:13.832078 30022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "01eeb4c96b1f4097ac1ef4dd96d57660"
format_stamp: "Formatted at 2026-08-06 20:27:13 on dist-test-slave-6k22"
server_key: "70591a849977b1c65438e7a280695dae"
server_key_iv: "ac1cae003d2c47f00f46e8fb60063b07"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:13.837857 30022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:27:13.858104 30046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:13.860122 30022 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.004s
I20260806 20:27:13.860332 30022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "01eeb4c96b1f4097ac1ef4dd96d57660"
format_stamp: "Formatted at 2026-08-06 20:27:13 on dist-test-slave-6k22"
server_key: "70591a849977b1c65438e7a280695dae"
server_key_iv: "ac1cae003d2c47f00f46e8fb60063b07"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:13.860590 30022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:27:13.885641 30022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:27:13.891007 30022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:27:13.891387 30022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:27:13.892628 30022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:27:13.894346 30022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:27:13.894444 30022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:13.894609 30022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:27:13.894699 30022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:13.941210 30022 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:45705
I20260806 20:27:13.941304 30165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:45705 every 8 connection(s)
I20260806 20:27:13.943028 30022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:27:13.950359 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 30022
I20260806 20:27:13.950577 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:27:13.951256 17847 external_mini_cluster.cc:1428] Setting key 5a7330aeb35d9bec7e12cd88aa437784
Aug 06 20:27:13 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048033, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:27:13.986404 30170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:13.947659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:37443 (local address 127.17.109.254:41967)
0806 20:27:13.964075 (+ 16416us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:13.964083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:13.964111 (+ 28us) server_negotiation.cc:407] Connection header received
0806 20:27:13.966269 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:13.966277 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:13.966385 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:13.966507 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:27:13.968472 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:13.969392 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:13.971496 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:13.972178 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:13.972328 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:13.977516 (+ 5188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:27:13.977525 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:27:13.977531 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:27:13.977582 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:27:13.981768 (+ 4186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:13.982896 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:13.982903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:13.982908 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:13.983115 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:13.983780 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:13.983787 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:13.983791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:13.984255 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:27:13.984437 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:13.985640 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0806 20:27:13.985923 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16256,"thread_start_us":16087,"threads_started":1}
I20260806 20:27:13.987903 30169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:13.946926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:41967 (local address 127.17.109.194:37443)
0806 20:27:13.949952 (+ 3026us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:13.950025 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 20:27:13.965700 (+ 15675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:27:13.966866 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:27:13.966880 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:27:13.967517 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:27:13.968248 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:13.968269 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:13.969572 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:13.969578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:13.971301 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:13.971321 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:13.972454 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:13.972462 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:13.972834 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:13.973763 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:27:13.973800 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:27:13.977219 (+ 3419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:27:13.981961 (+ 4742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:27:13.981981 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:27:13.982003 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:27:13.982719 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:27:13.983265 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:27:13.983273 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:27:13.983279 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:27:13.983621 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:27:13.984539 (+ 918us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:27:13.984549 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:27:13.985076 (+ 527us) client_negotiation.cc:769] Sending connection context
0806 20:27:13.985439 (+ 363us) client_negotiation.cc:240] Negotiation successful
0806 20:27:13.986170 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2688,"thread_start_us":2398,"threads_started":1}
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
I20260806 20:27:13.989614 30166 heartbeater.cc:347] Connected to a master server at 127.17.109.254:41967
I20260806 20:27:13.990013 30166 heartbeater.cc:464] Registering TS with master...
I20260806 20:27:13.991025 30166 heartbeater.cc:511] Master 127.17.109.254:41967 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
I20260806 20:27:13.993060 29783 ts_manager.cc:194] Registered new tserver with Master: 01eeb4c96b1f4097ac1ef4dd96d57660 (127.17.109.194:45705)
I20260806 20:27:13.994470 29783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:37443
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:27:14.028745 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33117/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41967
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:27:14.242154 30015 heartbeater.cc:503] Master 127.17.109.254:41967 was elected leader, sending a full tablet report...
W20260806 20:27:14.385691 30174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:27:14.385977 30174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:27:14.386180 30174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:27:14.393559 30174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:27:14.393674 30174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:27:14.393873 30174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:27:14.404181 30174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.109.212:33117/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:27:14.405575 30174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:27:14.406895 30174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:27:14.420518 30182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:27:14.420583 30179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:27:14.420696 30180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:27:14.421454 30174 server_base.cc:1034] running on GCE node
I20260806 20:27:14.422078 30174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:27:14.423105 30174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:27:14.424358 30174 hybrid_clock.cc:648] HybridClock initialized: now 1786048034424286 us; error 43 us; skew 500 ppm
I20260806 20:27:14.424744 30174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:27:14 dist-test-slave-6k22 krb5kdc[29275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048034, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:27:14.428135 30174 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:27:14.429533 30174 webserver.cc:460] Webserver started at http://127.17.109.195:38751/ using document root <none> and password file <none>
I20260806 20:27:14.430204 30174 fs_manager.cc:359] Metadata directory not provided
I20260806 20:27:14.430305 30174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:27:14.430562 30174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 20:27:14 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048034, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:27:14.446990 30174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "bd410a63508348e0b79232b4b5076f2a"
format_stamp: "Formatted at 2026-08-06 20:27:14 on dist-test-slave-6k22"
server_key: "bcfe7d38e2003f768444338cd0a0df38"
server_key_iv: "a2664cb4bb775c420213311926b1c7ec"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:14.447953 30174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "bd410a63508348e0b79232b4b5076f2a"
format_stamp: "Formatted at 2026-08-06 20:27:14 on dist-test-slave-6k22"
server_key: "bcfe7d38e2003f768444338cd0a0df38"
server_key_iv: "a2664cb4bb775c420213311926b1c7ec"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:14.454030 30174 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260806 20:27:14.470840 30190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:14.472846 30174 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 20:27:14.473058 30174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "bd410a63508348e0b79232b4b5076f2a"
format_stamp: "Formatted at 2026-08-06 20:27:14 on dist-test-slave-6k22"
server_key: "bcfe7d38e2003f768444338cd0a0df38"
server_key_iv: "a2664cb4bb775c420213311926b1c7ec"
server_key_version: "kuduclusterkey@0"
I20260806 20:27:14.473341 30174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:27:14.497476 30174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:27:14.502714 30174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:27:14.503058 30174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:27:14.504216 30174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:27:14.505839 30174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:27:14.505929 30174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:14.506047 30174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:27:14.506105 30174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:27:14.547209 30174 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:41103
I20260806 20:27:14.547271 30307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:41103 every 8 connection(s)
I20260806 20:27:14.548843 30174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:27:14.559274 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 30174
I20260806 20:27:14.559506 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:27:14.560083 17847 external_mini_cluster.cc:1428] Setting key 96d45712c82a155cae6e19a6fa8af512
Aug 06 20:27:14 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048034, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:27:14.571736 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.551726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:60157 (local address 127.17.109.254:41967)
0806 20:27:14.552206 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.552210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.553413 (+ 1203us) server_negotiation.cc:407] Connection header received
0806 20:27:14.555236 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.555241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.555335 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.555455 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:27:14.556892 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.557612 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.559201 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.559846 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.559981 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.564198 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:27:14.564203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:27:14.564206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:27:14.564240 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:27:14.567893 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:14.569038 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:14.569048 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:14.569051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:14.569195 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:14.569768 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:14.569773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:14.569776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:14.570138 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:27:14.570325 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.571051 (+ 726us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.571306 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":267,"threads_started":1}
I20260806 20:27:14.572989 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.552248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:41967 (local address 127.17.109.195:60157)
0806 20:27:14.553241 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:14.553286 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 20:27:14.554973 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:27:14.555716 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:27:14.555726 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:27:14.556147 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:27:14.556727 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.556744 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.557775 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.557782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.559051 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.559062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.560084 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.560091 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.560366 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.561111 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:27:14.561132 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:27:14.563971 (+ 2839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:27:14.568067 (+ 4096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:27:14.568078 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:27:14.568106 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:27:14.568873 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:27:14.569332 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:27:14.569336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:27:14.569340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:27:14.569581 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:27:14.570300 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:27:14.570308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:27:14.570652 (+ 344us) client_negotiation.cc:769] Sending connection context
0806 20:27:14.570956 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 20:27:14.571536 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":501,"threads_started":1}
I20260806 20:27:14.574826 30308 heartbeater.cc:347] Connected to a master server at 127.17.109.254:41967
I20260806 20:27:14.575281 30308 heartbeater.cc:464] Registering TS with master...
I20260806 20:27:14.576535 30308 heartbeater.cc:511] Master 127.17.109.254:41967 requested a full tablet report, sending...
I20260806 20:27:14.578305 29783 ts_manager.cc:194] Registered new tserver with Master: bd410a63508348e0b79232b4b5076f2a (127.17.109.195:41103)
I20260806 20:27:14.579326 29783 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:60157
I20260806 20:27:14.589852 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:27:14.608424 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.594538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54530 (local address 127.17.109.254:41967)
0806 20:27:14.594818 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.594827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.595254 (+ 427us) server_negotiation.cc:407] Connection header received
0806 20:27:14.595385 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.595388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.595469 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.595601 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:27:14.597708 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.598432 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.599999 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.600427 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.600569 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.603059 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:27:14.603064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:27:14.603067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:27:14.603096 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:27:14.605957 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:14.606689 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:14.606693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:14.606695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:14.606821 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:14.607171 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:14.607174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:14.607176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:14.607429 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:27:14.607740 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.607982 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.608153 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 20:27:14.612715 29783 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:27:14.620077 29783 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:27:14.656391 30321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.632639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:41103 (local address 127.0.0.1:45424)
0806 20:27:14.646181 (+ 13542us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:14.646234 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:27:14.646458 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:27:14.647934 (+ 1476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:27:14.647946 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:27:14.647998 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:27:14.649035 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.649052 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.650871 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.650876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.653445 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.653454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.655197 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.655213 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.655405 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.655490 (+ 85us) client_negotiation.cc:769] Sending connection context
0806 20:27:14.655642 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 20:27:14.655916 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13192,"mutex_wait_us":496,"spinlock_wait_cycles":1408,"thread_start_us":12907,"threads_started":1}
I20260806 20:27:14.656414 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.633034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.17.109.195:41103)
0806 20:27:14.645470 (+ 12436us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.645482 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.646417 (+ 935us) server_negotiation.cc:407] Connection header received
0806 20:27:14.647062 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.647067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.647300 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.647510 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:27:14.649269 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.650745 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.653611 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.654990 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.655144 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.655270 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.655743 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.655946 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12201,"thread_start_us":11997,"threads_started":1}
I20260806 20:27:14.657069 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.646847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45006)
0806 20:27:14.649296 (+ 2449us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:14.649319 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:27:14.649430 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:27:14.650039 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:27:14.650044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:27:14.650068 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:27:14.650550 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.650563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.652196 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.652201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.654321 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.654336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.656274 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.656280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.656482 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.656521 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 20:27:14.656622 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 20:27:14.656729 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2330,"mutex_wait_us":1,"thread_start_us":2177,"threads_started":1}
I20260806 20:27:14.657823 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.647242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.17.109.193:40163)
0806 20:27:14.648716 (+ 1474us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.648727 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.649543 (+ 816us) server_negotiation.cc:407] Connection header received
0806 20:27:14.649646 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.649653 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.649864 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.650045 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:27:14.650781 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.652003 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.654485 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.655736 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.655936 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.656100 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.657303 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.657482 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1243,"thread_start_us":1034,"threads_started":1}
I20260806 20:27:14.658294 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.632966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45705 (local address 127.0.0.1:50922)
0806 20:27:14.646188 (+ 13222us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:14.646235 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 20:27:14.646490 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:27:14.647652 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:27:14.647664 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:27:14.647707 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:27:14.648434 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.648448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.650475 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.650482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.653437 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.653452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.657437 (+ 3985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.657443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.657627 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.657677 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 20:27:14.657759 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 20:27:14.657909 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13088,"mutex_wait_us":133,"spinlock_wait_cycles":7040,"thread_start_us":12928,"threads_started":1}
I20260806 20:27:14.658973 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.633496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.17.109.194:45705)
0806 20:27:14.644545 (+ 11049us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.644560 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.646372 (+ 1812us) server_negotiation.cc:407] Connection header received
0806 20:27:14.646722 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.646731 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.647033 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.647245 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:27:14.648655 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.650333 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.655341 (+ 5008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.657168 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.657402 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.657563 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.658604 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.658739 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10792,"thread_start_us":11069,"threads_started":1}
I20260806 20:27:14.661232 30241 tablet_service.cc:1467] Processing CreateTablet for tablet 5b38cf47da9949b8ad264d8e3cf64d06 (DEFAULT_TABLE table=test-table [id=14c2c626414849b795137d6eef6e2d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:27:14.661494 29946 tablet_service.cc:1467] Processing CreateTablet for tablet 5b38cf47da9949b8ad264d8e3cf64d06 (DEFAULT_TABLE table=test-table [id=14c2c626414849b795137d6eef6e2d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:27:14.662858 29946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b38cf47da9949b8ad264d8e3cf64d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:27:14.662858 30241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b38cf47da9949b8ad264d8e3cf64d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:27:14.662663 30100 tablet_service.cc:1467] Processing CreateTablet for tablet 5b38cf47da9949b8ad264d8e3cf64d06 (DEFAULT_TABLE table=test-table [id=14c2c626414849b795137d6eef6e2d97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:27:14.663991 30100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b38cf47da9949b8ad264d8e3cf64d06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:27:14.676184 30330 tablet_bootstrap.cc:492] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: Bootstrap starting.
I20260806 20:27:14.679816 30330 tablet_bootstrap.cc:654] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: Neither blocks nor log segments found. Creating new log.
I20260806 20:27:14.680910 30330 log.cc:824] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: Log is configured to *not* fsync() on all Append() calls
I20260806 20:27:14.684026 30330 tablet_bootstrap.cc:492] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: No bootstrap required, opened a new log
I20260806 20:27:14.684312 30330 ts_tablet_manager.cc:1397] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260806 20:27:14.685106 30331 tablet_bootstrap.cc:492] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Bootstrap starting.
I20260806 20:27:14.688367 30332 tablet_bootstrap.cc:492] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: Bootstrap starting.
I20260806 20:27:14.689652 30331 tablet_bootstrap.cc:654] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Neither blocks nor log segments found. Creating new log.
I20260806 20:27:14.690738 30331 log.cc:824] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Log is configured to *not* fsync() on all Append() calls
I20260806 20:27:14.692250 30332 tablet_bootstrap.cc:654] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: Neither blocks nor log segments found. Creating new log.
I20260806 20:27:14.693571 30332 log.cc:824] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: Log is configured to *not* fsync() on all Append() calls
I20260806 20:27:14.695549 30330 raft_consensus.cc:348] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.696060 30331 tablet_bootstrap.cc:492] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: No bootstrap required, opened a new log
I20260806 20:27:14.696125 30330 raft_consensus.cc:374] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:27:14.696256 30330 raft_consensus.cc:729] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 173b2d3ab81046c6982c1323f8ccb7f5, State: Initialized, Role: FOLLOWER
I20260806 20:27:14.696385 30331 ts_tablet_manager.cc:1397] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260806 20:27:14.697041 30330 consensus_queue.cc:260] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.697356 30332 tablet_bootstrap.cc:492] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: No bootstrap required, opened a new log
I20260806 20:27:14.697716 30332 ts_tablet_manager.cc:1397] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260806 20:27:14.699368 30330 ts_tablet_manager.cc:1428] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260806 20:27:14.705859 30331 raft_consensus.cc:348] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.706516 30331 raft_consensus.cc:374] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:27:14.706629 30331 raft_consensus.cc:729] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd410a63508348e0b79232b4b5076f2a, State: Initialized, Role: FOLLOWER
I20260806 20:27:14.707358 30331 consensus_queue.cc:260] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.708993 30308 heartbeater.cc:503] Master 127.17.109.254:41967 was elected leader, sending a full tablet report...
I20260806 20:27:14.709784 30331 ts_tablet_manager.cc:1428] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260806 20:27:14.710421 30018 tablet.cc:2369] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:27:14.714109 30332 raft_consensus.cc:348] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.714851 30332 raft_consensus.cc:374] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:27:14.715029 30332 raft_consensus.cc:729] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01eeb4c96b1f4097ac1ef4dd96d57660, State: Initialized, Role: FOLLOWER
I20260806 20:27:14.715886 30332 consensus_queue.cc:260] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.717296 30166 heartbeater.cc:503] Master 127.17.109.254:41967 was elected leader, sending a full tablet report...
I20260806 20:27:14.718178 30332 ts_tablet_manager.cc:1428] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: Time spent starting tablet: real 0.020s user 0.011s sys 0.004s
W20260806 20:27:14.800927 30309 tablet.cc:2369] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:27:14.846922 30337 raft_consensus.cc:482] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:27:14.847249 30337 raft_consensus.cc:504] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.848943 30337 leader_election.cc:288] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01eeb4c96b1f4097ac1ef4dd96d57660 (127.17.109.194:45705), 173b2d3ab81046c6982c1323f8ccb7f5 (127.17.109.193:40163)
I20260806 20:27:14.855283 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.849597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:59277 (local address 127.17.109.194:45705)
0806 20:27:14.849838 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.849844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.849865 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:27:14.850130 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.850140 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.850259 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.850384 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:27:14.850930 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.851769 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.853584 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.854346 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.854535 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.854614 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.854893 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.855022 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 20:27:14.855305 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.849473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45705 (local address 127.17.109.195:59277)
0806 20:27:14.849756 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:14.849783 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:27:14.849926 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:27:14.850461 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:27:14.850466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:27:14.850503 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:27:14.850729 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.850741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.851900 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.851909 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.853414 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.853426 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.854536 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.854542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.854734 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.854759 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 20:27:14.854866 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 20:27:14.855002 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":45}
I20260806 20:27:14.856346 30120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b38cf47da9949b8ad264d8e3cf64d06" candidate_uuid: "bd410a63508348e0b79232b4b5076f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" is_pre_election: true
I20260806 20:27:14.856546 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.850119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:39721 (local address 127.17.109.193:40163)
0806 20:27:14.850354 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.850361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.850853 (+ 492us) server_negotiation.cc:407] Connection header received
0806 20:27:14.851032 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.851036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.851097 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.851211 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:27:14.851844 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.852537 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.854214 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.854994 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.855180 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.855250 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.856163 (+ 913us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.856298 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 20:27:14.856652 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.849939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.17.109.195:39721)
0806 20:27:14.850740 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:14.850762 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:27:14.850879 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:27:14.851205 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:27:14.851209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:27:14.851240 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:27:14.851688 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.851699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.852690 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.852696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.854048 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:27:14.854059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.855198 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.855205 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:27:14.855419 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.855469 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 20:27:14.856135 (+ 666us) client_negotiation.cc:240] Negotiation successful
0806 20:27:14.856274 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":3,"thread_start_us":455,"threads_started":1}
I20260806 20:27:14.856881 30120 raft_consensus.cc:2454] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd410a63508348e0b79232b4b5076f2a in term 0.
I20260806 20:27:14.857645 30193 leader_election.cc:302] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01eeb4c96b1f4097ac1ef4dd96d57660, bd410a63508348e0b79232b4b5076f2a; no voters:
I20260806 20:27:14.857697 29966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b38cf47da9949b8ad264d8e3cf64d06" candidate_uuid: "bd410a63508348e0b79232b4b5076f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" is_pre_election: true
I20260806 20:27:14.858099 30337 raft_consensus.cc:2781] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:27:14.858280 30337 raft_consensus.cc:482] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:27:14.858253 29966 raft_consensus.cc:2454] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd410a63508348e0b79232b4b5076f2a in term 0.
I20260806 20:27:14.858412 30337 raft_consensus.cc:3037] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:27:14.860841 30337 raft_consensus.cc:504] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.861433 30337 leader_election.cc:288] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [CANDIDATE]: Term 1 election: Requested vote from peers 01eeb4c96b1f4097ac1ef4dd96d57660 (127.17.109.194:45705), 173b2d3ab81046c6982c1323f8ccb7f5 (127.17.109.193:40163)
I20260806 20:27:14.861996 30120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b38cf47da9949b8ad264d8e3cf64d06" candidate_uuid: "bd410a63508348e0b79232b4b5076f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660"
I20260806 20:27:14.862053 29966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b38cf47da9949b8ad264d8e3cf64d06" candidate_uuid: "bd410a63508348e0b79232b4b5076f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "173b2d3ab81046c6982c1323f8ccb7f5"
I20260806 20:27:14.862277 30120 raft_consensus.cc:3037] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:27:14.862298 29966 raft_consensus.cc:3037] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:27:14.864682 30120 raft_consensus.cc:2454] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd410a63508348e0b79232b4b5076f2a in term 1.
I20260806 20:27:14.864775 29966 raft_consensus.cc:2454] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd410a63508348e0b79232b4b5076f2a in term 1.
I20260806 20:27:14.865221 30193 leader_election.cc:302] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01eeb4c96b1f4097ac1ef4dd96d57660, bd410a63508348e0b79232b4b5076f2a; no voters:
I20260806 20:27:14.865523 30337 raft_consensus.cc:2781] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:27:14.866106 30337 raft_consensus.cc:686] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 1 LEADER]: Becoming Leader. State: Replica: bd410a63508348e0b79232b4b5076f2a, State: Running, Role: LEADER
I20260806 20:27:14.866664 30337 consensus_queue.cc:237] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } }
I20260806 20:27:14.871470 29783 catalog_manager.cc:5495] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a reported cstate change: term changed from 0 to 1, leader changed from <none> to bd410a63508348e0b79232b4b5076f2a (127.17.109.195). New cstate: current_term: 1 leader_uuid: "bd410a63508348e0b79232b4b5076f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd410a63508348e0b79232b4b5076f2a" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 41103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 } health_report { overall_health: UNKNOWN } } }
I20260806 20:27:14.905404 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.899747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.17.109.195:41103)
0806 20:27:14.899975 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.899983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.900004 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:27:14.900236 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.900242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.900329 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.900499 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:27:14.900918 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.901636 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.903119 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.903545 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.903727 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.904273 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:27:14.904810 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:27:14.904977 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.905047 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.905133 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 20:27:14.913861 30120 raft_consensus.cc:1261] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 1 FOLLOWER]: Refusing update from remote peer bd410a63508348e0b79232b4b5076f2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:27:14.913893 29966 raft_consensus.cc:1261] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 1 FOLLOWER]: Refusing update from remote peer bd410a63508348e0b79232b4b5076f2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:27:14.914844 30340 consensus_queue.cc:1035] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "01eeb4c96b1f4097ac1ef4dd96d57660" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:27:14.915243 30337 consensus_queue.cc:1035] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "173b2d3ab81046c6982c1323f8ccb7f5" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 40163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:27:14.941521 29783 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:27:14.941830 29783 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:27:14.946193 29783 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f938b9d9781341eda5374f564c0fd67d: Sending DeleteTablet for 3 replicas of tablet 5b38cf47da9949b8ad264d8e3cf64d06
I20260806 20:27:14.947378 30241 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b38cf47da9949b8ad264d8e3cf64d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:27:14 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:45424
I20260806 20:27:14.947923 30100 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b38cf47da9949b8ad264d8e3cf64d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:27:14 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:50922
I20260806 20:27:14.948252 29946 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b38cf47da9949b8ad264d8e3cf64d06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:27:14 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:45006
I20260806 20:27:14.951092 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 29873
I20260806 20:27:14.959511 30354 tablet_replica.cc:330] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: stopping tablet replica
I20260806 20:27:14.960223 30353 tablet_replica.cc:330] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: stopping tablet replica
I20260806 20:27:14.960834 30354 raft_consensus.cc:2229] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:27:14.961905 30353 raft_consensus.cc:2229] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:27:14.963027 30353 raft_consensus.cc:2258] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:27:14.963660 30354 raft_consensus.cc:2258] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:27:14.964948 30356 tablet_replica.cc:330] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: stopping tablet replica
I20260806 20:27:14.965579 30356 raft_consensus.cc:2229] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:27:14.966205 30356 raft_consensus.cc:2258] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:27:14.968832 30356 ts_tablet_manager.cc:1905] T 5b38cf47da9949b8ad264d8e3cf64d06 P 173b2d3ab81046c6982c1323f8ccb7f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:27:14.971720 30354 ts_tablet_manager.cc:1905] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:27:14.973354 30353 ts_tablet_manager.cc:1905] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 20:27:14 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:27:14.984814 30354 ts_tablet_manager.cc:1918] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:27:14.985049 30354 log.cc:1196] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/5b38cf47da9949b8ad264d8e3cf64d06
I20260806 20:27:14.985610 30354 ts_tablet_manager.cc:1939] T 5b38cf47da9949b8ad264d8e3cf64d06 P bd410a63508348e0b79232b4b5076f2a: Deleting consensus metadata
I20260806 20:27:14.985661 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:14.953104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45022 (local address 127.17.109.193:40163)
0806 20:27:14.953354 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:14.953363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:14.964091 (+ 10728us) server_negotiation.cc:407] Connection header received
0806 20:27:14.964382 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:14.964391 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:14.964505 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:14.964631 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:27:14.966508 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.967818 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.970256 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:14.970932 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:14.971037 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:14.977174 (+ 6137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:27:14.977193 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:27:14.977212 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:27:14.977265 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:27:14.981539 (+ 4274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:14.982583 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:14.982593 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:14.982599 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:14.982769 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:14.983274 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:14.983279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:14.983283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:14.983851 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:27:14.984126 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:14.984773 (+ 647us) server_negotiation.cc:299] Negotiation successful
0806 20:27:14.985227 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 20:27:14.986320 30353 ts_tablet_manager.cc:1918] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:27:14.986501 30353 log.cc:1196] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/5b38cf47da9949b8ad264d8e3cf64d06
I20260806 20:27:14.987165 30353 ts_tablet_manager.cc:1939] T 5b38cf47da9949b8ad264d8e3cf64d06 P 01eeb4c96b1f4097ac1ef4dd96d57660: Deleting consensus metadata
I20260806 20:27:14.987836 30009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:27:14.989176 29767 catalog_manager.cc:4841] TS bd410a63508348e0b79232b4b5076f2a (127.17.109.195:41103): tablet 5b38cf47da9949b8ad264d8e3cf64d06 (table test-table [id=14c2c626414849b795137d6eef6e2d97]) successfully deleted
I20260806 20:27:14.989681 29769 catalog_manager.cc:4841] TS 01eeb4c96b1f4097ac1ef4dd96d57660 (127.17.109.194:45705): tablet 5b38cf47da9949b8ad264d8e3cf64d06 (table test-table [id=14c2c626414849b795137d6eef6e2d97]) successfully deleted
I20260806 20:27:15.112421 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 29873
W20260806 20:27:15.133093 29769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.109.193:40163 (error 108)
I20260806 20:27:15.133193 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 30022
I20260806 20:27:15.135576 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.135023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45036)
0806 20:27:15.135239 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:15.135375 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 20:27:15.135855 29769 catalog_manager.cc:4584] TS 173b2d3ab81046c6982c1323f8ccb7f5 (127.17.109.193:40163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b38cf47da9949b8ad264d8e3cf64d06: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Aug 06 20:27:15 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:27:15.151185 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.134070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.17.109.194:45705)
0806 20:27:15.134275 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:15.134281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:15.134382 (+ 101us) server_negotiation.cc:407] Connection header received
0806 20:27:15.134537 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:15.134543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:15.134628 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:15.134744 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:27:15.136422 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:15.137591 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:15.138891 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:15.139525 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:15.139865 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:15.144336 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:27:15.144358 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:27:15.144381 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:27:15.144429 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:27:15.148312 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:15.149070 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:15.149080 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:15.149084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:15.149258 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:15.149612 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:15.149617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:15.149621 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:15.150081 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:27:15.150379 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:15.150568 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 20:27:15.150791 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:27:15.152119 30161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:27:15.189688 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.189161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45040)
0806 20:27:15.189398 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:15.189507 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260806 20:27:15.190876 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.190468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45044)
0806 20:27:15.190648 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:15.190736 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 20:27:15.243862 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 30022
I20260806 20:27:15.251657 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.250990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45052)
0806 20:27:15.251273 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:15.251403 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 20:27:15.252880 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.252451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45062)
0806 20:27:15.252629 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:15.252726 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 20:27:15.264688 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 30174
Aug 06 20:27:15 dist-test-slave-6k22 krb5kdc[29275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:27:15.283865 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.265572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.17.109.195:41103)
0806 20:27:15.265774 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 20:27:15.265780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:27:15.265809 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:27:15.266037 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:27:15.266042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:27:15.266144 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:27:15.266278 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:27:15.267597 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:15.268811 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:15.270117 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:27:15.270875 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:27:15.271022 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:27:15.275898 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:27:15.275913 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:27:15.275932 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:27:15.275981 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:27:15.280691 (+ 4710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:15.281446 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:15.281462 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:15.281465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:15.281670 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:27:15.282076 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:27:15.282081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:27:15.282085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:27:15.282595 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:27:15.282938 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0806 20:27:15.283138 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 20:27:15.283402 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 20:27:15.284791 30303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:27:15.334112 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.333631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45064)
0806 20:27:15.333863 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:15.333958 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260806 20:27:15.335265 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.334853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45066)
0806 20:27:15.335058 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:15.335148 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260806 20:27:15.409004 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 30174
I20260806 20:27:15.430706 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 29749
I20260806 20:27:15.432056 29820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:27:15.547891 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:27:15.547167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:40163 (local address 127.0.0.1:45076)
0806 20:27:15.547366 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 20:27:15.547514 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:40163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 20:27:15.547909 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 29749
I20260806 20:27:15.573125 17847 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 20:27:15.908533 17847 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 20:27:15.908912 17847 mini_ranger.cc:66] Stopping Ranger...
I20260806 20:27:16.264751 17847 mini_ranger.cc:68] Stopped Ranger
2026-08-06 20:27:16.265 UTC [29306] LOG: received smart shutdown request
2026-08-06 20:27:16.269 UTC [29306] LOG: background worker "logical replication launcher" (PID 29317) exited with exit code 1
2026-08-06 20:27:16.269 UTC [29312] LOG: shutting down
2026-08-06 20:27:16.449 UTC [29306] LOG: database system is shut down
2026-08-06T20:27:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53028 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:27:16 dist-test-slave-6k22 krb5kdc[30371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:27:16 dist-test-slave-6k22 krb5kdc[30371](info): set up 2 sockets
Aug 06 20:27:16 dist-test-slave-6k22 krb5kdc[30371](info): commencing operation
krb5kdc: starting...
W20260806 20:27:17.793107 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.008s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:27:17 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:27:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:27:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.109.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.109.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/rangeradmin_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/rangeradmin_127.17.109.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.109.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.109.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/rangerlookup_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/rangerlookup_127.17.109.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.109.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.109.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/HTTP_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/HTTP_127.17.109.212@KRBTEST.COM.keytab.
I20260806 20:27:18.206918 17847 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskcvsjCP/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/postgres -l logfile start
2026-08-06 20:27:20.875 UTC [30403] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 20:27:20.875 UTC [30403] LOG: listening on IPv4 address "127.17.109.212", port 58417
2026-08-06 20:27:20.879 UTC [30403] LOG: listening on Unix socket "/tmp/.s.PGSQL.58417"
2026-08-06 20:27:20.902 UTC [30406] LOG: database system was shut down at 2026-08-06 20:27:19 UTC
2026-08-06 20:27:20.906 UTC [30403] LOG: database system is ready to accept connections
I20260806 20:27:21.927912 17847 mini_postgres.cc:96] Postgres bound to 58417
2026-08-06 20:27:21.950 UTC [30414] FATAL: database "slave" does not exist
127.17.109.212:58417 - accepting connections
I20260806 20:27:21.951236 17847 mini_ranger.cc:161] Starting Ranger...
I20260806 20:27:21.982208 17847 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 20:27:22.484436 17847 mini_ranger.cc:87] Created ranger Postgres database
I20260806 20:27:22.484578 17847 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-admin
2026-08-06 20:27:22,795 [I] DB FLAVOR :POSTGRES
2026-08-06 20:27:22,796 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 20:27:22,796 [I] Checking connection..
2026-08-06 20:27:22,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 20:27:23,059 [I] Checking connection passed.
2026-08-06 20:27:23,060 [I] --------- Verifying version history table ---------
2026-08-06 20:27:23,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 20:27:23,307 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 20:27:23,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 20:27:23,545 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 20:27:23,545 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 20:27:23,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 20:27:23.813 UTC [30481] WARNING: there is no transaction in progress
2026-08-06 20:27:23,820 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 20:27:23,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 20:27:24,101 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 20:27:24,101 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 20:27:24,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 20:27:24,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 20:27:24,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-6k22.c.gcp-upstream.internal','N') ;"
2026-08-06 20:27:24,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 20:27:25,172 [I] Table x_portal_user does not exist in database ranger
2026-08-06 20:27:25,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 20:27:25,441 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 20:27:25,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 20:27:25,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 20:27:25,968 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 20:27:25,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 20:27:26.956 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:26.973 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:26.989 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.006 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.023 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.042 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.100 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.108 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.123 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.134 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.144 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.159 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.168 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.175 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.181 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27.189 UTC [30599] WARNING: there is no transaction in progress
2026-08-06 20:27:27,879 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 20:27:27,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 20:27:28,128 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 20:27:28,129 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 20:27:28,400 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 20:27:28,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 20:27:28,668 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 20:27:28,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 20:27:28,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 20:27:29,183 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-6k22.c.gcp-upstream.internal';"
2026-08-06 20:27:29,453 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 20:27:29,453 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 20:27:29,749 [I] DB_PATCHES have already been applied
I20260806 20:27:29.755394 17847 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-admin:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskcvsjCP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 20:27:29.755499 17847 mini_ranger.cc:193] Using host: 127.17.109.212
I20260806 20:27:29.775779 17847 mini_ranger.cc:227] Ranger admin URL: http://127.17.109.212:58805
Aug 06, 2026 8:27:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 8:27:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 8:27:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 8:27:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 8:27:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 8:27:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 8:27:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 20:27:31.029866 17847 mini_ranger.cc:160] Time spent starting Ranger: real 9.079s user 0.008s sys 0.053s
Aug 06, 2026 8:27:52 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 8:27:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 06, 2026 8:27:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 8:27:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 8:27:53 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 20:27:53.715943 17847 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.109.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.109.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/rangerkms_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.109.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/rangerkms_127.17.109.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.109.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.109.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.109.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/HTTP_127.17.109.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.109.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/HTTP_127.17.109.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 20:27:53 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048073, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 20:27:53.928596 17847 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 20:27:53.954770 17847 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 20:27:54.873675 17847 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 20:27:54.873816 17847 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms
I20260806 20:27:54.873884 17847 mini_ranger_kms.cc:227] Using postgres at 127.17.109.212:58417
2026-08-06 20:27:55,262 [I] DB FLAVOR :POSTGRES
2026-08-06 20:27:55,263 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 20:27:55,263 [I] Checking connection
2026-08-06 20:27:55,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 20:27:55,538 [I] connection success
2026-08-06 20:27:55,539 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 20:27:55,539 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 20:27:55,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 20:27:55,854 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 20:27:55,855 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 20:27:55,855 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 20:27:55,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.109.212:58417/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 20:27:56,115 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 20:27:56.618314 17847 mini_ranger_kms.cc:310] Created kms service
I20260806 20:27:56.853202 17847 mini_ranger_kms.cc:326] Created kudu user
I20260806 20:27:56.900178 17847 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 20:27:56 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048073, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:27:57.360492 17847 mini_ranger_kms.cc:384] Added ranger policy
I20260806 20:27:57.360822 17847 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms:/tmp/dist-test-taskcvsjCP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskcvsjCP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 20:27:57.360863 17847 mini_ranger_kms.cc:240] Using host: 127.17.109.212
I20260806 20:27:57.381918 17847 mini_ranger_kms.cc:276] Ranger KMS PID: 30790
I20260806 20:27:57.382081 17847 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.109.212:33319
Aug 06, 2026 8:27:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 8:27:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 8:27:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 8:27:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 8:27:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 8:27:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 20:27:59 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786048079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.109.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:27:59.781430 17847 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.853s user 0.067s sys 0.354s
I20260806 20:27:59.781726 17847 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 20:27:59.781926 17847 mini_ranger_kms.cc:401] 127.17.109.212:33319/kms/v1/keys
Aug 06 20:28:00 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.109.212@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
Aug 06 20:28:00 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048079, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.109.212@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
Aug 06 20:28:02 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:28:02.784042 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38297
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:38297
--ranger_config_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33319/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:03.137276 30842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:03.137598 30842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:03.137758 30842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:03.145910 30842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:03.146020 30842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:03.146099 30842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:03.146209 30842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:03.146277 30842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:03.156824 30842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.109.212:33319/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:38297
--ranger_config_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:03.158239 30842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:03.159673 30842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:03.173952 30850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:03.173835 30848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:03.174180 30847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:03.174578 30842 server_base.cc:1034] running on GCE node
I20260806 20:28:03.175948 30842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:03.177426 30842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:03.178664 30842 hybrid_clock.cc:648] HybridClock initialized: now 1786048083178624 us; error 43 us; skew 500 ppm
I20260806 20:28:03.179064 30842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:03 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048083, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:03.187717 30842 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:28:03.189344 30842 webserver.cc:460] Webserver started at http://127.17.109.254:40097/ using document root <none> and password file <none>
I20260806 20:28:03.190017 30842 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:03.190151 30842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:03.190471 30842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 20:28:03 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048083, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:03.260085 30842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "45e3e573854049e78b0718cdef70c6a1"
format_stamp: "Formatted at 2026-08-06 20:28:03 on dist-test-slave-6k22"
server_key: "da9e70ef7b249f69e63f96b4a96cd802"
server_key_iv: "e7ddf962f72420ecef700e24eb1f8d8b"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:03.261058 30842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "45e3e573854049e78b0718cdef70c6a1"
format_stamp: "Formatted at 2026-08-06 20:28:03 on dist-test-slave-6k22"
server_key: "da9e70ef7b249f69e63f96b4a96cd802"
server_key_iv: "e7ddf962f72420ecef700e24eb1f8d8b"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:03.267053 30842 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260806 20:28:03.288859 30858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:03.290578 30842 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 20:28:03.290808 30842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "45e3e573854049e78b0718cdef70c6a1"
format_stamp: "Formatted at 2026-08-06 20:28:03 on dist-test-slave-6k22"
server_key: "da9e70ef7b249f69e63f96b4a96cd802"
server_key_iv: "e7ddf962f72420ecef700e24eb1f8d8b"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:03.291054 30842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:03.318724 30842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:03.323033 30842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:03.323388 30842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:03.343853 30842 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:38297
I20260806 20:28:03.343853 30912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:38297 every 8 connection(s)
I20260806 20:28:03.345363 30842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:28:03.349917 30913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:03.355048 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 30842
I20260806 20:28:03.355372 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:28:03.356160 17847 external_mini_cluster.cc:1428] Setting key f0b45ac5510eb543cc15bc9e8346f228
I20260806 20:28:03.364537 30913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Bootstrap starting.
I20260806 20:28:03.368929 30913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:03.370437 30913 log.cc:824] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:03.377658 30913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: No bootstrap required, opened a new log
I20260806 20:28:03.387049 30913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } }
I20260806 20:28:03.387480 30913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:03.387634 30913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e3e573854049e78b0718cdef70c6a1, State: Initialized, Role: FOLLOWER
I20260806 20:28:03.388332 30913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } }
I20260806 20:28:03.388579 30913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 06 20:28:03 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:03.388796 30913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:28:03.389019 30913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:03.392405 30913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } }
I20260806 20:28:03.393115 30913 leader_election.cc:302] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45e3e573854049e78b0718cdef70c6a1; no voters:
I20260806 20:28:03.393862 30913 leader_election.cc:288] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:28:03.394062 30918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:03.396920 30913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:03.401746 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:03.358553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59390 (local address 127.17.109.254:38297)
0806 20:28:03.373121 (+ 14568us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:03.373140 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:03.373516 (+ 376us) server_negotiation.cc:407] Connection header received
0806 20:28:03.375985 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:03.376014 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:03.376545 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:03.377080 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:03.378962 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:03.381138 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:03.383110 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:03.384304 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:03.384502 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:03.390070 (+ 5568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:03.390086 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:03.390106 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:03.390163 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:03.394646 (+ 4483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:03.395814 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:03.395824 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:03.395833 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:03.396049 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:03.396624 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:03.396629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:03.396632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:03.397432 (+ 800us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:03.397729 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:03.399362 (+ 1633us) server_negotiation.cc:299] Negotiation successful
0806 20:28:03.399935 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14243,"thread_start_us":14456,"threads_started":1}
I20260806 20:28:03.402516 30918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 1 LEADER]: Becoming Leader. State: Replica: 45e3e573854049e78b0718cdef70c6a1, State: Running, Role: LEADER
I20260806 20:28:03.403223 30918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } }
I20260806 20:28:03.408610 30920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45e3e573854049e78b0718cdef70c6a1. Latest consensus state: current_term: 1 leader_uuid: "45e3e573854049e78b0718cdef70c6a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } } }
I20260806 20:28:03.409109 30920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:03.409068 30919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45e3e573854049e78b0718cdef70c6a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } } }
I20260806 20:28:03.409410 30919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:03.417516 30925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:28:03.424046 30925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:28:03.443020 30913 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 20:28:03.447074 30925 catalog_manager.cc:1347] Generated new cluster ID: 9afbb7e8b1d2404ea4361e12e469f1b4
I20260806 20:28:03.447341 30925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:28:03.479439 30925 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:28:03.481665 30925 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:28:03.495209 30925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Generated new TSK 0
I20260806 20:28:03.496699 30925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:28:04.379990 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38297
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:28:04 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786048084, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 20:28:04.835148 30960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:04.835593 30960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:04.835832 30960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:04.847671 30960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:04.847819 30960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:04.848061 30960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:28:04.863997 30960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.109.212:33319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:04.865927 30960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:04.867795 30960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:04.896126 30972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:04.897446 30971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:04.901520 30960 server_base.cc:1034] running on GCE node
W20260806 20:28:04.902622 30974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:04.903498 30960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:04.904947 30960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:04.906265 30960 hybrid_clock.cc:648] HybridClock initialized: now 1786048084906187 us; error 73 us; skew 500 ppm
I20260806 20:28:04.906692 30960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:04 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048084, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:04.917941 30960 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:28:04.919996 30960 webserver.cc:460] Webserver started at http://127.17.109.193:33063/ using document root <none> and password file <none>
I20260806 20:28:04.920815 30960 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:04.920950 30960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:04.921298 30960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 20:28:04 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048084, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:04.951370 30960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "13082b2594a44613ba394394fef4b47c"
format_stamp: "Formatted at 2026-08-06 20:28:04 on dist-test-slave-6k22"
server_key: "343a0e12b854bb9187762dbaba81e4b0"
server_key_iv: "831d39d8176320f557970d27ba217f6f"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:04.952787 30960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "13082b2594a44613ba394394fef4b47c"
format_stamp: "Formatted at 2026-08-06 20:28:04 on dist-test-slave-6k22"
server_key: "343a0e12b854bb9187762dbaba81e4b0"
server_key_iv: "831d39d8176320f557970d27ba217f6f"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:04.964079 30960 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.007s sys 0.001s
I20260806 20:28:04.997965 30981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:05.001684 30960 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.005s
I20260806 20:28:05.002022 30960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "13082b2594a44613ba394394fef4b47c"
format_stamp: "Formatted at 2026-08-06 20:28:04 on dist-test-slave-6k22"
server_key: "343a0e12b854bb9187762dbaba81e4b0"
server_key_iv: "831d39d8176320f557970d27ba217f6f"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:05.002396 30960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:05.070142 30960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:05.076201 30960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:05.076560 30960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:05.077907 30960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:05.080276 30960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:05.080385 30960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:05.080569 30960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:05.080647 30960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:28:05 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048084, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:05.193105 30960 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:36313
I20260806 20:28:05.193151 31098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:36313 every 8 connection(s)
I20260806 20:28:05.194869 30960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:28:05.204352 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 30960
I20260806 20:28:05.204572 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:28:05.205432 17847 external_mini_cluster.cc:1428] Setting key 1e102438927e91bbad5c079090abce9a
Aug 06 20:28:05 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048084, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:05.239693 31101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:05.197864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:39799 (local address 127.17.109.254:38297)
0806 20:28:05.218106 (+ 20242us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:05.218112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:05.218135 (+ 23us) server_negotiation.cc:407] Connection header received
0806 20:28:05.220099 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:05.220105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:05.220207 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:05.220298 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:05.222781 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:05.223970 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:05.226459 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:05.227088 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:05.227187 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:05.231961 (+ 4774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:05.231968 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:05.231973 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:05.232013 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:05.235631 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:05.236601 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:05.236607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:05.236611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:05.236787 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:05.237368 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:05.237373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:05.237377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:05.237764 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:05.237895 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:05.239003 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0806 20:28:05.239234 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20093,"thread_start_us":14056,"threads_started":1}
I20260806 20:28:05.242245 31102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:05.198808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38297 (local address 127.17.109.193:39799)
0806 20:28:05.207866 (+ 9058us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:05.207919 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:05.219737 (+ 11818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:05.220931 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:05.220947 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:05.221638 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:05.222553 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:05.222584 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:05.224197 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:05.224202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:05.226290 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:05.226307 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:05.227574 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:05.227583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:05.227899 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:05.228768 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:05.228801 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:05.231681 (+ 2880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:05.235830 (+ 4149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:05.235838 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:05.235857 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:05.236458 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:05.236942 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:05.236947 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:05.236950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:05.237226 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:05.238008 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:05.238014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:05.238523 (+ 509us) client_negotiation.cc:769] Sending connection context
0806 20:28:05.240269 (+ 1746us) client_negotiation.cc:240] Negotiation successful
0806 20:28:05.240848 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8295,"thread_start_us":11600,"threads_started":1}
I20260806 20:28:05.244349 31099 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38297
I20260806 20:28:05.244701 31099 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:05.245550 31099 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
I20260806 20:28:05.248692 30876 ts_manager.cc:194] Registered new tserver with Master: 13082b2594a44613ba394394fef4b47c (127.17.109.193:36313)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
I20260806 20:28:05.252772 30876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:39799
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:28:05.296038 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38297
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:28:05 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048084, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
W20260806 20:28:05.766279 31105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:05.766604 31105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:05.766782 31105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:05.777731 31105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:05.777936 31105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:05.778173 31105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:28:05.790390 31105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.109.212:33319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:05.791941 31105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:05.793462 31105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:05.811095 31120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:05.811436 31119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:05.811525 31122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:05.813359 31105 server_base.cc:1034] running on GCE node
I20260806 20:28:05.814002 31105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:05.815047 31105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:05.816303 31105 hybrid_clock.cc:648] HybridClock initialized: now 1786048085816249 us; error 41 us; skew 500 ppm
I20260806 20:28:05.816679 31105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:05 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:05.820624 31105 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:28:05.822438 31105 webserver.cc:460] Webserver started at http://127.17.109.194:44953/ using document root <none> and password file <none>
I20260806 20:28:05.823247 31105 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:05.823421 31105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:05.823840 31105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 20:28:05 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:05.843021 31105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3"
format_stamp: "Formatted at 2026-08-06 20:28:05 on dist-test-slave-6k22"
server_key: "71b338accb63bee4f1a8c8888ad25a51"
server_key_iv: "86d170c618e12e4ff96979929d50031b"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:05.844269 31105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3"
format_stamp: "Formatted at 2026-08-06 20:28:05 on dist-test-slave-6k22"
server_key: "71b338accb63bee4f1a8c8888ad25a51"
server_key_iv: "86d170c618e12e4ff96979929d50031b"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:05.851667 31105 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260806 20:28:05.879333 31129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:05.881669 31105 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.009s
I20260806 20:28:05.881958 31105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3"
format_stamp: "Formatted at 2026-08-06 20:28:05 on dist-test-slave-6k22"
server_key: "71b338accb63bee4f1a8c8888ad25a51"
server_key_iv: "86d170c618e12e4ff96979929d50031b"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:05.882319 31105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:05.910099 31105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:05.916069 31105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:05.916469 31105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:05.917789 31105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:05.919824 31105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:05.919971 31105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:05.920132 31105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:05.920226 31105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:05.963757 31105 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:37415
I20260806 20:28:05.963799 31249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:37415 every 8 connection(s)
I20260806 20:28:05.965718 31105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:28:05.973068 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 31105
I20260806 20:28:05.973259 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:28:05.973821 17847 external_mini_cluster.cc:1428] Setting key 5b991286e14994cedb82e2a2a0f8707b
Aug 06 20:28:06 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048085, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
I20260806 20:28:06.017946 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:05.970128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:39843 (local address 127.17.109.254:38297)
0806 20:28:05.995301 (+ 25173us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:05.995307 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:05.995332 (+ 25us) server_negotiation.cc:407] Connection header received
0806 20:28:05.997776 (+ 2444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:05.997783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:05.997914 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:05.998003 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:06.000430 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.001606 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.004141 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.004790 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.004904 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:06.009812 (+ 4908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:06.009819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:06.009837 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:06.009884 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:06.013748 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.014785 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.014791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.014795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.014956 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.015533 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.015538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.015542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.015948 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:06.016213 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:06.017291 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0806 20:28:06.017545 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25041,"thread_start_us":27546,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
I20260806 20:28:06.020532 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:05.969460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38297 (local address 127.17.109.194:39843)
0806 20:28:05.994208 (+ 24748us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:05.994282 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:05.997414 (+ 3132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:05.998598 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:05.998616 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:05.999264 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:06.000176 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:06.000207 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.001834 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.001844 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:06.003969 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:06.003984 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.005283 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.005291 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:06.005616 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:06.006505 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:06.006545 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:06.009522 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:06.013975 (+ 4453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:06.013984 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:06.014005 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:06.014644 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:06.015115 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:06.015120 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:06.015124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:06.015388 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:06.016313 (+ 925us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:06.016322 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:06.016779 (+ 457us) client_negotiation.cc:769] Sending connection context
0806 20:28:06.018314 (+ 1535us) client_negotiation.cc:240] Negotiation successful
0806 20:28:06.018924 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24359,"thread_start_us":25615,"threads_started":1}
I20260806 20:28:06.022809 31250 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38297
I20260806 20:28:06.023119 31250 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:06.023970 31250 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
I20260806 20:28:06.025664 30876 ts_manager.cc:194] Registered new tserver with Master: 4fbd7e39aa964ed7b5cbec2b24297cc3 (127.17.109.194:37415)
I20260806 20:28:06.026722 30876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:39843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:28:06.060424 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38297
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:28:06.259459 31099 heartbeater.cc:503] Master 127.17.109.254:38297 was elected leader, sending a full tablet report...
W20260806 20:28:06.455658 31257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:06.455966 31257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:06.456135 31257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:06.463315 31257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:06.463485 31257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:06.463686 31257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:28:06.473495 31257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.109.212:33319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:06.475003 31257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:06.476416 31257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:06.490924 31263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:06.491019 31262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:06.490909 31265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:06.491842 31257 server_base.cc:1034] running on GCE node
I20260806 20:28:06.492509 31257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:06.493629 31257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:06.494886 31257 hybrid_clock.cc:648] HybridClock initialized: now 1786048086494820 us; error 52 us; skew 500 ppm
I20260806 20:28:06.495277 31257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:06 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:06.499014 31257 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:28:06.500655 31257 webserver.cc:460] Webserver started at http://127.17.109.195:44271/ using document root <none> and password file <none>
I20260806 20:28:06.501442 31257 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:06.501605 31257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:06.501955 31257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 20:28:06 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:06.520851 31257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50"
format_stamp: "Formatted at 2026-08-06 20:28:06 on dist-test-slave-6k22"
server_key: "ea7fc17e6ce2cb06b70b235e6eb0567e"
server_key_iv: "8f5b870be09011aa95d679f43ebebf23"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:06.521961 31257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50"
format_stamp: "Formatted at 2026-08-06 20:28:06 on dist-test-slave-6k22"
server_key: "ea7fc17e6ce2cb06b70b235e6eb0567e"
server_key_iv: "8f5b870be09011aa95d679f43ebebf23"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:06.529212 31257 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260806 20:28:06.548257 31272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:06.550186 31257 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 20:28:06.550390 31257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50"
format_stamp: "Formatted at 2026-08-06 20:28:06 on dist-test-slave-6k22"
server_key: "ea7fc17e6ce2cb06b70b235e6eb0567e"
server_key_iv: "8f5b870be09011aa95d679f43ebebf23"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:06.550674 31257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:06.586206 31257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:06.590978 31257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:06.591279 31257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:06.592305 31257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:06.593756 31257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:06.593837 31257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:06.593927 31257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:06.593981 31257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:06.631237 31257 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:42393
I20260806 20:28:06.631273 31391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:42393 every 8 connection(s)
I20260806 20:28:06.632606 31257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:28:06.638300 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 31257
I20260806 20:28:06.638507 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:28:06.639150 17847 external_mini_cluster.cc:1428] Setting key c055eb5446c8e12c9d210974449a7c54
Aug 06 20:28:06 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048086, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:06.654569 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:06.635225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:57911 (local address 127.17.109.254:38297)
0806 20:28:06.635821 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:06.635826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:06.636816 (+ 990us) server_negotiation.cc:407] Connection header received
0806 20:28:06.639000 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:06.639005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:06.639070 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:06.639169 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:06.640947 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.641871 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.643819 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.644342 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.644491 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:06.648098 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:06.648103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:06.648107 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:06.648142 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:06.650984 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.651809 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.651814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.651816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.651952 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.652350 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.652353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.652355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.652658 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:06.652792 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:06.653937 (+ 1145us) server_negotiation.cc:299] Negotiation successful
0806 20:28:06.654250 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":270,"threads_started":1}
I20260806 20:28:06.655787 31395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:06.635914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38297 (local address 127.17.109.195:57911)
0806 20:28:06.636627 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:06.636664 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:06.638692 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:06.639437 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:06.639449 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:06.639981 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:06.640736 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:06.640766 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.642037 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.642045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:06.643655 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:06.643669 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.644590 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.644597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:06.644875 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:06.645628 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:06.645652 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:06.647886 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:06.651116 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:06.651123 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:06.651136 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:06.651681 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:06.652049 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:06.652052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:06.652055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:06.652245 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:06.652884 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:06.652895 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:06.653342 (+ 447us) client_negotiation.cc:769] Sending connection context
0806 20:28:06.653779 (+ 437us) client_negotiation.cc:240] Negotiation successful
0806 20:28:06.654393 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":292,"threads_started":1}
I20260806 20:28:06.657305 31392 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38297
I20260806 20:28:06.657661 31392 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:06.658521 31392 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
I20260806 20:28:06.660101 30876 ts_manager.cc:194] Registered new tserver with Master: 2f049a9ec0094bb6a700d1d6d6cd4e50 (127.17.109.195:42393)
I20260806 20:28:06.661016 30876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:57911
I20260806 20:28:06.668198 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:28:06.670023 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 30960
Aug 06 20:28:06 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:28:06.688138 31397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:06.671029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52200 (local address 127.17.109.193:36313)
0806 20:28:06.671782 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:06.671792 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:06.671827 (+ 35us) server_negotiation.cc:407] Connection header received
0806 20:28:06.671884 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:06.671890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:06.672143 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:06.672362 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:06.673997 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.675013 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.676501 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.677053 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.677203 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:06.681781 (+ 4578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:06.681794 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:06.681809 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:06.681845 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:06.685053 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.685820 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.685828 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.685830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.685972 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.686377 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.686381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.686382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.686755 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:06.687087 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:06.687401 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 20:28:06.687755 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":326,"threads_started":1}
I20260806 20:28:06.689641 31094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:06.775484 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 30960
I20260806 20:28:06.794274 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 31105
Aug 06 20:28:06 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:28:06.811896 31399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:06.795291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.17.109.194:37415)
0806 20:28:06.795949 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:06.795956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:06.795983 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:28:06.796055 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:06.796062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:06.796305 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:06.796459 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:06.798098 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.799149 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.800588 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.801176 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.801286 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:06.806179 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:06.806190 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:06.806203 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:06.806236 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:06.809240 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.809920 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.809927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.809929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.810078 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.810408 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.810412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.810414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.810747 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:06.810972 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:06.811279 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 20:28:06.811483 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":336,"threads_started":1}
I20260806 20:28:06.813494 31243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:06.896663 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 31105
I20260806 20:28:06.921316 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 31257
Aug 06 20:28:06 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048082, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:28:06.938828 31401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:06.922504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.17.109.195:42393)
0806 20:28:06.923176 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:06.923182 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:06.923211 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:28:06.923295 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:06.923302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:06.923509 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:06.923701 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:06.925148 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.926151 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.927529 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:06.928152 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:06.928304 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:06.933029 (+ 4725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:06.933040 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:06.933056 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:06.933096 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:06.936079 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.936760 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.936767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.936769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.936918 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:06.937271 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:06.937275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:06.937277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:06.937622 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:06.937920 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:06.938137 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 20:28:06.938336 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":369,"threads_started":1}
I20260806 20:28:06.940261 31387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:07.026297 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 31257
I20260806 20:28:07.044790 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 30842
I20260806 20:28:07.046159 30908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:07.154559 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 30842
I20260806 20:28:07.186898 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38297
--webserver_interface=127.17.109.254
--webserver_port=40097
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:38297
--ranger_config_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33319/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.109.212:33319/kms with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:07.526242 31406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:07.526537 31406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:07.526625 31406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:07.533964 31406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:07.534088 31406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:07.534138 31406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:07.534197 31406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:07.534245 31406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:07.544567 31406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.109.212:33319/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:38297
--ranger_config_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:38297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=40097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:07.546078 31406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:07.547384 31406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:07.559777 31412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:07.559828 31414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:07.559892 31411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:07.560443 31406 server_base.cc:1034] running on GCE node
I20260806 20:28:07.561206 31406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:07.562605 31406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:07.563843 31406 hybrid_clock.cc:648] HybridClock initialized: now 1786048087563783 us; error 58 us; skew 500 ppm
I20260806 20:28:07.564195 31406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:07 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048087, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:07.567677 31406 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:28:07.569264 31406 webserver.cc:460] Webserver started at http://127.17.109.254:40097/ using document root <none> and password file <none>
I20260806 20:28:07.569928 31406 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:07.570077 31406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 20:28:07 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048087, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:07.626348 31406 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 20:28:07.629752 31422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:07.631230 31406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:28:07.631462 31406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "45e3e573854049e78b0718cdef70c6a1"
format_stamp: "Formatted at 2026-08-06 20:28:03 on dist-test-slave-6k22"
server_key: "da9e70ef7b249f69e63f96b4a96cd802"
server_key_iv: "e7ddf962f72420ecef700e24eb1f8d8b"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:07.632526 31406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:07.645754 31406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:07.649930 31406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:07.650352 31406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:07.669660 31406 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:38297
I20260806 20:28:07.669709 31475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:38297 every 8 connection(s)
I20260806 20:28:07.671156 31406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:28:07.678329 31476 sys_catalog.cc:263] Verifying existing consensus state
I20260806 20:28:07.678781 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 31406
I20260806 20:28:07.679726 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:36313
--local_ip_for_outbound_sockets=127.17.109.193
--tserver_master_addrs=127.17.109.254:38297
--webserver_port=33063
--webserver_interface=127.17.109.193
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.109.212:33319/kms with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:28:07.681829 31476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Bootstrap starting.
I20260806 20:28:07.701263 31476 log.cc:824] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:07.710320 31476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:28:07.710903 31476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Bootstrap complete.
I20260806 20:28:07.720060 31476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } }
I20260806 20:28:07.720948 31476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45e3e573854049e78b0718cdef70c6a1, State: Initialized, Role: FOLLOWER
I20260806 20:28:07.721616 31476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } }
I20260806 20:28:07.721861 31476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:28:07.721992 31476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:28:07.722174 31476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 1 FOLLOWER]: Advancing to term 2
I20260806 20:28:07.726037 31476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } }
I20260806 20:28:07.726522 31476 leader_election.cc:302] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45e3e573854049e78b0718cdef70c6a1; no voters:
I20260806 20:28:07.727188 31476 leader_election.cc:288] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 20:28:07.727361 31481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 20:28:07.728703 31481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [term 2 LEADER]: Becoming Leader. State: Replica: 45e3e573854049e78b0718cdef70c6a1, State: Running, Role: LEADER
I20260806 20:28:07.729357 31481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } }
I20260806 20:28:07.729537 31476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:07.732214 31482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "45e3e573854049e78b0718cdef70c6a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } } }
I20260806 20:28:07.732681 31482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:07.735069 31483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45e3e573854049e78b0718cdef70c6a1. Latest consensus state: current_term: 2 leader_uuid: "45e3e573854049e78b0718cdef70c6a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45e3e573854049e78b0718cdef70c6a1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 38297 } } }
I20260806 20:28:07.735513 31483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1 [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:07.737144 31487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:28:07.744490 31487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:28:07.746367 31487 catalog_manager.cc:1259] Loaded cluster ID: 9afbb7e8b1d2404ea4361e12e469f1b4
I20260806 20:28:07.746557 31487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:28:07.751672 31487 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:28:07.753635 31487 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Loaded TSK: 0
I20260806 20:28:07.755698 31487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:28:07.759804 31476 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260806 20:28:08.138283 31479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:08.138796 31479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:08.139015 31479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:08.150858 31479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:08.151010 31479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:08.151240 31479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:28:08.167230 31479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.109.212:33319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:36313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=33063
--tserver_master_addrs=127.17.109.254:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:08.169236 31479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:08.171106 31479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:08.230746 31516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:08.230782 31515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:08.235147 31479 server_base.cc:1034] running on GCE node
W20260806 20:28:08.235680 31518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:08.236658 31479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:08.238476 31479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:08.240362 31479 hybrid_clock.cc:648] HybridClock initialized: now 1786048088239853 us; error 502 us; skew 500 ppm
I20260806 20:28:08.240768 31479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:08 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048088, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:08.251149 31479 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:28:08.261219 31479 webserver.cc:460] Webserver started at http://127.17.109.193:33063/ using document root <none> and password file <none>
I20260806 20:28:08.262287 31479 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:08.262537 31479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 20:28:08 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048088, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:08.318082 31479 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.004s sys 0.008s
I20260806 20:28:08.327929 31528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:08.329689 31479 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.007s sys 0.001s
I20260806 20:28:08.329890 31479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "13082b2594a44613ba394394fef4b47c"
format_stamp: "Formatted at 2026-08-06 20:28:04 on dist-test-slave-6k22"
server_key: "343a0e12b854bb9187762dbaba81e4b0"
server_key_iv: "831d39d8176320f557970d27ba217f6f"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:08.331187 31479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:08.379158 31479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:08.384487 31479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:08.384802 31479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:08.386088 31479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:08.388037 31479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:08.388154 31479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:08.388278 31479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:08.388370 31479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:08.445194 31479 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:36313
I20260806 20:28:08.445242 31648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:36313 every 8 connection(s)
I20260806 20:28:08.446851 31479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:28:08.453032 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 31479
I20260806 20:28:08.453867 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:37415
--local_ip_for_outbound_sockets=127.17.109.194
--tserver_master_addrs=127.17.109.254:38297
--webserver_port=44953
--webserver_interface=127.17.109.194
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.109.212:33319/kms with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:28:08 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048088, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:08.492818 31651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:08.454184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38297 (local address 127.17.109.193:38627)
0806 20:28:08.455125 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:08.455206 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:08.458089 (+ 2883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:08.468771 (+ 10682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:08.468786 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:08.469492 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:08.470285 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:08.470315 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:08.472858 (+ 2543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:08.472867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:08.475177 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:08.475197 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:08.476905 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:08.476915 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:08.477271 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:08.478171 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:08.478205 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:08.481397 (+ 3192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:08.486757 (+ 5360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:08.486770 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:08.486790 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:08.487543 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:08.488197 (+ 654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:08.488204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:08.488210 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:08.488499 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:08.489701 (+ 1202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:08.489711 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:08.490205 (+ 494us) client_negotiation.cc:769] Sending connection context
0806 20:28:08.490556 (+ 351us) client_negotiation.cc:240] Negotiation successful
0806 20:28:08.491221 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":1315,"threads_started":1}
I20260806 20:28:08.496361 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:08.455292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:38627 (local address 127.17.109.254:38297)
0806 20:28:08.466750 (+ 11458us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:08.466775 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:08.466834 (+ 59us) server_negotiation.cc:407] Connection header received
0806 20:28:08.467397 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:08.467423 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:08.468026 (+ 603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:08.469231 (+ 1205us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:08.470556 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:08.472628 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:08.475428 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:08.476639 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:08.477819 (+ 1180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:08.481739 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:08.481752 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:08.481776 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:08.481831 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:08.486542 (+ 4711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:08.487798 (+ 1256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:08.487810 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:08.487822 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:08.488037 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:08.488683 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:08.488690 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:08.488696 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:08.489520 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:08.494260 (+ 4740us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:08.494551 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 20:28:08.495014 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11135,"thread_start_us":11881,"threads_started":1}
I20260806 20:28:08.499866 31649 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38297
I20260806 20:28:08.500182 31649 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:08.501070 31649 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
W20260806 20:28:08.504717 31649 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:38297 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 20:28:08.504937 31649 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:08.505215 31649 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
I20260806 20:28:08.506525 31649 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:08.506899 31649 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
W20260806 20:28:08.508029 31649 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 20:28:08.878011 31653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:08.878392 31653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:08.878589 31653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:08.886361 31653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:08.886549 31653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:08.886742 31653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:28:08.900565 31653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.109.212:33319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:37415
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=44953
--tserver_master_addrs=127.17.109.254:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:08.902371 31653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:08.904191 31653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:08.921562 31668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:08.921936 31653 server_base.cc:1034] running on GCE node
W20260806 20:28:08.922441 31667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:08.922943 31671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:08.923735 31653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:08.924911 31653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:08.926203 31653 hybrid_clock.cc:648] HybridClock initialized: now 1786048088926141 us; error 51 us; skew 500 ppm
I20260806 20:28:08.926556 31653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:08 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048088, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:08.930778 31653 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:28:08.932752 31653 webserver.cc:460] Webserver started at http://127.17.109.194:44953/ using document root <none> and password file <none>
I20260806 20:28:08.933629 31653 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:08.933808 31653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 20:28:08 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048088, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:08.973631 31653 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.008s sys 0.001s
I20260806 20:28:08.978012 31681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:08.980078 31653 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260806 20:28:08.980324 31653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3"
format_stamp: "Formatted at 2026-08-06 20:28:05 on dist-test-slave-6k22"
server_key: "71b338accb63bee4f1a8c8888ad25a51"
server_key_iv: "86d170c618e12e4ff96979929d50031b"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:08.981954 31653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:09.021935 31653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:09.027798 31653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:09.028126 31653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:09.029505 31653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:09.031491 31653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:09.031631 31653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 20:28:09.031769 31653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:09.031863 31653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:28:09 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786048089, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:09.085014 31653 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:37415
I20260806 20:28:09.085086 31796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:37415 every 8 connection(s)
I20260806 20:28:09.087141 31653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:28:09.100083 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 31653
I20260806 20:28:09.100915 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.109.212:33319/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:42393
--local_ip_for_outbound_sockets=127.17.109.195
--tserver_master_addrs=127.17.109.254:38297
--webserver_port=44271
--webserver_interface=127.17.109.195
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.109.212:33319/kms with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:28:09 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048088, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:09.120498 31799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:09.090361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:46375 (local address 127.17.109.254:38297)
0806 20:28:09.095163 (+ 4802us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:09.095168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:09.101000 (+ 5832us) server_negotiation.cc:407] Connection header received
0806 20:28:09.101090 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:09.101095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:09.101246 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:09.101366 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:09.103530 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:09.104808 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:09.107294 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:09.108013 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:09.108115 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:09.112649 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:09.112657 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:09.112662 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:09.112704 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:09.116552 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:09.117550 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:09.117556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:09.117560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:09.117718 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:09.118295 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:09.118300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:09.118304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:09.118717 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:09.118852 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:09.119881 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0806 20:28:09.120120 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4676,"thread_start_us":4545,"threads_started":1}
I20260806 20:28:09.123080 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:09.090930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38297 (local address 127.17.109.194:46375)
0806 20:28:09.096384 (+ 5454us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:09.096462 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:09.100129 (+ 3667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:09.101875 (+ 1746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:09.101885 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:09.102343 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:09.103230 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:09.103251 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:09.105028 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:09.105034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:09.107099 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:09.107115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:09.108505 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:09.108512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:09.108829 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:09.109648 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:09.109680 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:09.112371 (+ 2691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:09.116735 (+ 4364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:09.116744 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:09.116763 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:09.117396 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:09.117879 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:09.117884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:09.117888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:09.118137 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:09.118964 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:09.118972 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:09.119408 (+ 436us) client_negotiation.cc:769] Sending connection context
0806 20:28:09.121081 (+ 1673us) client_negotiation.cc:240] Negotiation successful
0806 20:28:09.121682 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5041,"thread_start_us":12682,"threads_started":1}
I20260806 20:28:09.125180 31797 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38297
I20260806 20:28:09.125502 31797 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:09.126396 31797 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
I20260806 20:28:09.128533 31439 ts_manager.cc:194] Registered new tserver with Master: 4fbd7e39aa964ed7b5cbec2b24297cc3 (127.17.109.194:37415)
I20260806 20:28:09.130702 31439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:46375
I20260806 20:28:09.508210 31649 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:09.508983 31649 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
I20260806 20:28:09.510466 31439 ts_manager.cc:194] Registered new tserver with Master: 13082b2594a44613ba394394fef4b47c (127.17.109.193:36313)
I20260806 20:28:09.512034 31439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:38627
Aug 06 20:28:09 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048089, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
W20260806 20:28:09.623791 31801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:09.624292 31801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:09.624482 31801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:09.638021 31801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:09.638214 31801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:09.638465 31801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:28:09.654824 31801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.109.212:33319/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:42393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=44271
--tserver_master_addrs=127.17.109.254:38297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:09.656522 31801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:09.658134 31801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:09.674841 31807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:09.688649 31808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:09.701205 31810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:09.702436 31801 server_base.cc:1034] running on GCE node
I20260806 20:28:09.703351 31801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:09.704617 31801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:09.705921 31801 hybrid_clock.cc:648] HybridClock initialized: now 1786048089705851 us; error 70 us; skew 500 ppm
I20260806 20:28:09.706337 31801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:09 dist-test-slave-6k22 krb5kdc[30371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048089, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:09.710417 31801 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:28:09.721325 31801 webserver.cc:460] Webserver started at http://127.17.109.195:44271/ using document root <none> and password file <none>
I20260806 20:28:09.722414 31801 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:09.722596 31801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 20:28:09 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048089, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:09.752018 31801 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.010s sys 0.000s
I20260806 20:28:09.756206 31818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:09.758185 31801 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 20:28:09.758496 31801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50"
format_stamp: "Formatted at 2026-08-06 20:28:06 on dist-test-slave-6k22"
server_key: "ea7fc17e6ce2cb06b70b235e6eb0567e"
server_key_iv: "8f5b870be09011aa95d679f43ebebf23"
server_key_version: "kuduclusterkey@0"
I20260806 20:28:09.760268 31801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:09.789605 31801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:09.795469 31801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:09.795893 31801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:09.797309 31801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:09.799399 31801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:09.799527 31801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:09.799729 31801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:09.799852 31801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 20:28:09 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048089, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for HTTP/127.17.109.212@KRBTEST.COM
I20260806 20:28:09.999929 31801 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:42393
I20260806 20:28:10.000329 31935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:42393 every 8 connection(s)
I20260806 20:28:10.002259 31801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:28:10.002683 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 31801
I20260806 20:28:10.049540 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.003857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49752 (local address 127.17.109.254:38297)
0806 20:28:10.016024 (+ 12167us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.016031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.023627 (+ 7596us) server_negotiation.cc:407] Connection header received
0806 20:28:10.023731 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.023737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.023833 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.023985 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:10.026310 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.028106 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.032951 (+ 4845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.033969 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.034110 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.039664 (+ 5554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:10.039674 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:10.039682 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:10.039753 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:10.044285 (+ 4532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.046038 (+ 1753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.046047 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.046053 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.046271 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.047705 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.047710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.047713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.048127 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:10.048349 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.048895 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.049139 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12018,"thread_start_us":11870,"threads_started":1}
Aug 06 20:28:10 dist-test-slave-6k22 krb5kdc[30371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048089, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:10.069410 31942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.022139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:53031 (local address 127.17.109.254:38297)
0806 20:28:10.045908 (+ 23769us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.045917 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.047619 (+ 1702us) server_negotiation.cc:407] Connection header received
0806 20:28:10.052410 (+ 4791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.052415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.052489 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.052570 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:10.054392 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.055190 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.056905 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.057341 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.057477 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.061252 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:10.061258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:10.061261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:10.061294 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:10.065252 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.066240 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.066245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.066249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.066421 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.067410 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.067415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.067420 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.067837 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:10.067995 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.068759 (+ 764us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.068996 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23580,"thread_start_us":23410,"threads_started":1}
I20260806 20:28:10.070565 31941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.021222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:38297 (local address 127.17.109.195:53031)
0806 20:28:10.047435 (+ 26213us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.047494 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:10.052066 (+ 4572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:10.053090 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:10.053100 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:10.053581 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:10.054182 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.054199 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.055380 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.055384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.056742 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.056754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.057596 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.057604 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.057845 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.058586 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:10.058614 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:10.060966 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:10.065451 (+ 4485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:10.065460 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:10.065587 (+ 127us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:10.066089 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:10.066569 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:10.066574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:10.066578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:10.066943 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:10.067977 (+ 1034us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:10.067985 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:10.068370 (+ 385us) client_negotiation.cc:769] Sending connection context
0806 20:28:10.068660 (+ 290us) client_negotiation.cc:240] Negotiation successful
0806 20:28:10.069245 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25854,"thread_start_us":25564,"threads_started":1}
I20260806 20:28:10.072074 31936 heartbeater.cc:347] Connected to a master server at 127.17.109.254:38297
I20260806 20:28:10.072417 31936 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:10.073176 31936 heartbeater.cc:511] Master 127.17.109.254:38297 requested a full tablet report, sending...
I20260806 20:28:10.074723 31439 ts_manager.cc:194] Registered new tserver with Master: 2f049a9ec0094bb6a700d1d6d6cd4e50 (127.17.109.195:42393)
I20260806 20:28:10.075547 31439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:53031
I20260806 20:28:10.080610 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:28:10.121412 31942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.107234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.17.109.254:38297)
0806 20:28:10.107427 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.107432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.107956 (+ 524us) server_negotiation.cc:407] Connection header received
0806 20:28:10.108186 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.108190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.108269 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.108394 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:10.109948 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.110713 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.112216 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.112655 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.112796 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.115750 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:10.115759 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:10.115767 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:10.115811 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:10.118616 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.119339 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.119349 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.119351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.119494 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.119952 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.119962 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.119964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.120238 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:10.120503 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.120845 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.121081 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 20:28:10.126966 31439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 20:28:10.134729 31797 heartbeater.cc:503] Master 127.17.109.254:38297 was elected leader, sending a full tablet report...
W20260806 20:28:10.134979 31439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:28:10.183030 31961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.150640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:42393 (local address 127.0.0.1:43406)
0806 20:28:10.167476 (+ 16836us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.167498 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:10.167751 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:10.169076 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:10.169087 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:10.169130 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:10.169698 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.169715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.175639 (+ 5924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.175646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.179529 (+ 3883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.179545 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.182056 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.182065 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.182281 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.182338 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 20:28:10.182450 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 20:28:10.182655 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16712,"thread_start_us":16564,"threads_started":1}
I20260806 20:28:10.184489 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.151192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.17.109.195:42393)
0806 20:28:10.166367 (+ 15175us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.166381 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.167625 (+ 1244us) server_negotiation.cc:407] Connection header received
0806 20:28:10.168009 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.168022 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.168395 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.168636 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:10.169963 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.172166 (+ 2203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.180545 (+ 8379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.181810 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.183683 (+ 1873us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.183860 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.183931 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.184107 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14958,"thread_start_us":14789,"threads_started":1}
I20260806 20:28:10.184736 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.157411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.17.109.193:36313)
0806 20:28:10.175510 (+ 18099us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.175519 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.175550 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:28:10.175691 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.175700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.175971 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.176173 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:10.177358 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.179046 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.181628 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.183186 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.183378 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.183594 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.184144 (+ 550us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.184316 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17913,"thread_start_us":18944,"threads_started":1}
I20260806 20:28:10.185639 31964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.156676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49892)
0806 20:28:10.174206 (+ 17530us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.174226 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:10.174349 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:10.176592 (+ 2243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:10.176598 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:10.176626 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:10.177107 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.177120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.179239 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.179244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.181477 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.181492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.183750 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.183756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.183928 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.183960 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 20:28:10.184055 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 20:28:10.184171 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17399,"thread_start_us":16883,"threads_started":1}
I20260806 20:28:10.188660 31870 tablet_service.cc:1467] Processing CreateTablet for tablet 7d84d30e57034775b189e98142fa7bab (DEFAULT_TABLE table=test-table [id=a86854043eed48d0818cb8c07cf7799e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:10.189424 31580 tablet_service.cc:1467] Processing CreateTablet for tablet 7d84d30e57034775b189e98142fa7bab (DEFAULT_TABLE table=test-table [id=a86854043eed48d0818cb8c07cf7799e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:10.190145 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.150469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:37415 (local address 127.0.0.1:59920)
0806 20:28:10.151260 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.151302 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:10.151500 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:10.182895 (+ 31395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:10.182901 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:10.182929 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:10.183290 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.183302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.185510 (+ 2208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.185515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.187400 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.187416 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.189328 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.189335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.189520 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.189553 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 20:28:10.189655 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 20:28:10.189823 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":845,"threads_started":1}
I20260806 20:28:10.190316 31870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d84d30e57034775b189e98142fa7bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:10.190208 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.151175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.17.109.194:37415)
0806 20:28:10.182066 (+ 30891us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.182075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.182105 (+ 30us) server_negotiation.cc:407] Connection header received
0806 20:28:10.182200 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.182209 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.182461 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.183517 (+ 1056us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:10.183787 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.185343 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.187679 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.189095 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.189300 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.189465 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.189725 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.189877 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30712,"thread_start_us":31431,"threads_started":1}
I20260806 20:28:10.190959 31580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d84d30e57034775b189e98142fa7bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:10.193914 31730 tablet_service.cc:1467] Processing CreateTablet for tablet 7d84d30e57034775b189e98142fa7bab (DEFAULT_TABLE table=test-table [id=a86854043eed48d0818cb8c07cf7799e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:10.195223 31730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d84d30e57034775b189e98142fa7bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:10.202487 31966 tablet_bootstrap.cc:492] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Bootstrap starting.
I20260806 20:28:10.205785 31966 tablet_bootstrap.cc:654] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:10.206192 31967 tablet_bootstrap.cc:492] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: Bootstrap starting.
I20260806 20:28:10.208287 31966 log.cc:824] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:10.209826 31967 tablet_bootstrap.cc:654] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:10.211061 31967 log.cc:824] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:10.211741 31966 tablet_bootstrap.cc:492] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: No bootstrap required, opened a new log
I20260806 20:28:10.212025 31966 ts_tablet_manager.cc:1397] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 20:28:10.212143 31969 tablet_bootstrap.cc:492] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Bootstrap starting.
I20260806 20:28:10.213956 31967 tablet_bootstrap.cc:492] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: No bootstrap required, opened a new log
I20260806 20:28:10.214232 31967 ts_tablet_manager.cc:1397] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260806 20:28:10.215497 31969 tablet_bootstrap.cc:654] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:10.216765 31969 log.cc:824] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:10.220188 31969 tablet_bootstrap.cc:492] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: No bootstrap required, opened a new log
I20260806 20:28:10.220458 31969 ts_tablet_manager.cc:1397] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.006s
I20260806 20:28:10.222015 31967 raft_consensus.cc:348] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.222391 31967 raft_consensus.cc:374] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:10.222532 31967 raft_consensus.cc:729] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13082b2594a44613ba394394fef4b47c, State: Initialized, Role: FOLLOWER
I20260806 20:28:10.223138 31967 consensus_queue.cc:260] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.223399 31966 raft_consensus.cc:348] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.223904 31966 raft_consensus.cc:374] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:10.224051 31966 raft_consensus.cc:729] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f049a9ec0094bb6a700d1d6d6cd4e50, State: Initialized, Role: FOLLOWER
I20260806 20:28:10.224285 31649 heartbeater.cc:503] Master 127.17.109.254:38297 was elected leader, sending a full tablet report...
I20260806 20:28:10.224758 31967 ts_tablet_manager.cc:1428] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: Time spent starting tablet: real 0.010s user 0.007s sys 0.005s
I20260806 20:28:10.224774 31966 consensus_queue.cc:260] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.225787 31936 heartbeater.cc:503] Master 127.17.109.254:38297 was elected leader, sending a full tablet report...
I20260806 20:28:10.228195 31966 ts_tablet_manager.cc:1428] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260806 20:28:10.232322 31969 raft_consensus.cc:348] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.232687 31969 raft_consensus.cc:374] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:10.232807 31969 raft_consensus.cc:729] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fbd7e39aa964ed7b5cbec2b24297cc3, State: Initialized, Role: FOLLOWER
I20260806 20:28:10.233371 31969 consensus_queue.cc:260] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.234993 31969 ts_tablet_manager.cc:1428] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260806 20:28:10.274379 31940 tablet.cc:2369] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:28:10.353049 31798 tablet.cc:2369] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:28:10.365000 31972 raft_consensus.cc:482] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:10.365315 31973 raft_consensus.cc:482] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:10.365371 31972 raft_consensus.cc:504] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.365612 31973 raft_consensus.cc:504] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.367630 31973 leader_election.cc:288] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fbd7e39aa964ed7b5cbec2b24297cc3 (127.17.109.194:37415), 13082b2594a44613ba394394fef4b47c (127.17.109.193:36313)
I20260806 20:28:10.367630 31972 leader_election.cc:288] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fbd7e39aa964ed7b5cbec2b24297cc3 (127.17.109.194:37415), 2f049a9ec0094bb6a700d1d6d6cd4e50 (127.17.109.195:42393)
I20260806 20:28:10.377987 31941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.367958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:37415 (local address 127.17.109.195:48361)
0806 20:28:10.368305 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.368331 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:10.368470 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:10.369038 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:10.369046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:10.369086 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:10.369347 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.369361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.371638 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.371646 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.374391 (+ 2745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.374412 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.376900 (+ 2488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.376909 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.377201 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.377270 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 20:28:10.377384 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 20:28:10.377585 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260806 20:28:10.378929 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.368322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:48361 (local address 127.17.109.194:37415)
0806 20:28:10.368524 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.368531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.368552 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:28:10.368630 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.368636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.368747 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.368847 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:10.369548 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.371010 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.374655 (+ 3645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.376450 (+ 1795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.376680 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.376789 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.378364 (+ 1575us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.378587 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 20:28:10.380064 31751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d84d30e57034775b189e98142fa7bab" candidate_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" is_pre_election: true
I20260806 20:28:10.380738 31751 raft_consensus.cc:2454] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f049a9ec0094bb6a700d1d6d6cd4e50 in term 0.
I20260806 20:28:10.381630 31821 leader_election.cc:302] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f049a9ec0094bb6a700d1d6d6cd4e50, 4fbd7e39aa964ed7b5cbec2b24297cc3; no voters:
I20260806 20:28:10.382305 31973 raft_consensus.cc:2781] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:28:10.382459 31973 raft_consensus.cc:482] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:28:10.382568 31973 raft_consensus.cc:3037] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:10.386086 31973 raft_consensus.cc:504] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.387414 31751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d84d30e57034775b189e98142fa7bab" candidate_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3"
I20260806 20:28:10.387667 31751 raft_consensus.cc:3037] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:10.387871 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.368904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:46971 (local address 127.17.109.193:36313)
0806 20:28:10.369358 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.369367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.380988 (+ 11621us) server_negotiation.cc:407] Connection header received
0806 20:28:10.381052 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.381056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.381118 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.381194 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:10.381754 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.382976 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.385364 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.386639 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.386780 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.386855 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.387459 (+ 604us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.387608 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260806 20:28:10.387913 31941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.368526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.17.109.195:46971)
0806 20:28:10.379372 (+ 10846us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.379393 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:10.379533 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:10.381352 (+ 1819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:10.381359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:10.381387 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:10.381572 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.381583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.383167 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.383171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.385175 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.385191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.386979 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.386986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.387266 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.387305 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 20:28:10.387422 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 20:28:10.387552 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10752,"mutex_wait_us":3,"thread_start_us":17220,"threads_started":1}
I20260806 20:28:10.388692 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.368956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:59127 (local address 127.17.109.194:37415)
0806 20:28:10.379200 (+ 10244us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.379207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.380479 (+ 1272us) server_negotiation.cc:407] Connection header received
0806 20:28:10.380943 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.380948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.381018 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.381109 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:10.382130 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.383259 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.386138 (+ 2879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.387341 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.387521 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.387619 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.388253 (+ 634us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.388390 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10130,"threads_started":1}
I20260806 20:28:10.389694 31973 leader_election.cc:288] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [CANDIDATE]: Term 1 election: Requested vote from peers 4fbd7e39aa964ed7b5cbec2b24297cc3 (127.17.109.194:37415), 13082b2594a44613ba394394fef4b47c (127.17.109.193:36313)
I20260806 20:28:10.390112 31976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.367945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:37415 (local address 127.17.109.193:59127)
0806 20:28:10.380358 (+ 12413us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.380384 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:10.380767 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:10.381353 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:10.381361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:10.381395 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:10.381958 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.381974 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.383458 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.383466 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.386004 (+ 2538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.386017 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.387860 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.387867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.388048 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.388089 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 20:28:10.388210 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 20:28:10.388350 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12262,"mutex_wait_us":234,"spinlock_wait_cycles":6016,"thread_start_us":12332,"threads_started":1}
I20260806 20:28:10.390324 31599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d84d30e57034775b189e98142fa7bab" candidate_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13082b2594a44613ba394394fef4b47c"
I20260806 20:28:10.390277 31600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d84d30e57034775b189e98142fa7bab" candidate_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13082b2594a44613ba394394fef4b47c" is_pre_election: true
I20260806 20:28:10.390766 31599 raft_consensus.cc:3037] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:10.390838 31750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d84d30e57034775b189e98142fa7bab" candidate_uuid: "13082b2594a44613ba394394fef4b47c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" is_pre_election: true
I20260806 20:28:10.391763 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.371303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:46113 (local address 127.17.109.195:42393)
0806 20:28:10.371519 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.371528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.380677 (+ 9149us) server_negotiation.cc:407] Connection header received
0806 20:28:10.380900 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.380907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.380982 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.381066 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:10.382034 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.383158 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.386184 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.388374 (+ 2190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.388519 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.388580 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.391355 (+ 2775us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.391489 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":863,"server-negotiator.queue_time_us":74}
I20260806 20:28:10.392383 31751 raft_consensus.cc:2454] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f049a9ec0094bb6a700d1d6d6cd4e50 in term 1.
I20260806 20:28:10.392668 31975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.368115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:42393 (local address 127.17.109.193:46113)
0806 20:28:10.380557 (+ 12442us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.380582 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:10.380761 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:10.381298 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:10.381305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:10.381330 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:10.381799 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.381817 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.383317 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.383323 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.385713 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:10.385738 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.390811 (+ 5073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.390823 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:10.391101 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.391154 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 20:28:10.392002 (+ 848us) client_negotiation.cc:240] Negotiation successful
0806 20:28:10.392161 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12315,"mutex_wait_us":49,"spinlock_wait_cycles":6016,"thread_start_us":12368,"threads_started":1}
I20260806 20:28:10.392848 31821 leader_election.cc:302] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f049a9ec0094bb6a700d1d6d6cd4e50, 4fbd7e39aa964ed7b5cbec2b24297cc3; no voters:
I20260806 20:28:10.393142 31973 raft_consensus.cc:2781] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:10.394043 31599 raft_consensus.cc:2454] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f049a9ec0094bb6a700d1d6d6cd4e50 in term 1.
I20260806 20:28:10.393925 31890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d84d30e57034775b189e98142fa7bab" candidate_uuid: "13082b2594a44613ba394394fef4b47c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" is_pre_election: true
I20260806 20:28:10.394479 31973 raft_consensus.cc:686] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 1 LEADER]: Becoming Leader. State: Replica: 2f049a9ec0094bb6a700d1d6d6cd4e50, State: Running, Role: LEADER
I20260806 20:28:10.395045 31973 consensus_queue.cc:237] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } }
I20260806 20:28:10.399284 31531 leader_election.cc:302] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13082b2594a44613ba394394fef4b47c; no voters: 2f049a9ec0094bb6a700d1d6d6cd4e50, 4fbd7e39aa964ed7b5cbec2b24297cc3
I20260806 20:28:10.399724 31972 raft_consensus.cc:2726] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 20:28:10.400168 31439 catalog_manager.cc:5495] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f049a9ec0094bb6a700d1d6d6cd4e50 (127.17.109.195). New cstate: current_term: 1 leader_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f049a9ec0094bb6a700d1d6d6cd4e50" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 42393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 } health_report { overall_health: UNKNOWN } } }
I20260806 20:28:10.418036 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.412582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.17.109.195:42393)
0806 20:28:10.412819 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.412823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.412841 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:28:10.412984 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.412988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.413067 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.413220 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:28:10.413617 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.414508 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.415942 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.416367 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.416466 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.416920 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:28:10.417533 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:28:10.417651 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.417731 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.417800 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260806 20:28:10.426757 31599 raft_consensus.cc:1261] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 1 FOLLOWER]: Refusing update from remote peer 2f049a9ec0094bb6a700d1d6d6cd4e50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:10.426755 31751 raft_consensus.cc:1261] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 1 FOLLOWER]: Refusing update from remote peer 2f049a9ec0094bb6a700d1d6d6cd4e50: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:10.427668 31979 consensus_queue.cc:1035] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13082b2594a44613ba394394fef4b47c" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:10.428017 31973 consensus_queue.cc:1035] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fbd7e39aa964ed7b5cbec2b24297cc3" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 37415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:10.439721 31982 mvcc.cc:205] Tried to move back new op lower bound from 7315652978377867264 to 7315652978262568960. Current Snapshot: MvccSnapshot[applied={T|T < 7315652978377867264}]
I20260806 20:28:10.441159 31983 mvcc.cc:205] Tried to move back new op lower bound from 7315652978377867264 to 7315652978262568960. Current Snapshot: MvccSnapshot[applied={T|T < 7315652978377867264}]
I20260806 20:28:10.463428 31438 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:28:10.463790 31438 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:28:10.469180 31438 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45e3e573854049e78b0718cdef70c6a1: Sending DeleteTablet for 3 replicas of tablet 7d84d30e57034775b189e98142fa7bab
I20260806 20:28:10.470408 31580 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d84d30e57034775b189e98142fa7bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:10 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:49892
I20260806 20:28:10.470417 31870 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d84d30e57034775b189e98142fa7bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:10 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:43406
I20260806 20:28:10.470964 31730 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d84d30e57034775b189e98142fa7bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:10 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:59920
I20260806 20:28:10.471623 31729 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d84d30e57034775b189e98142fa7bab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:10 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:59920
I20260806 20:28:10.479866 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 31479
I20260806 20:28:10.488137 31994 tablet_replica.cc:330] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: stopping tablet replica
I20260806 20:28:10.488729 31994 raft_consensus.cc:2229] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:28:10.489396 31994 raft_consensus.cc:2258] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:10.492579 31993 tablet_replica.cc:330] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: stopping tablet replica
I20260806 20:28:10.493059 31993 raft_consensus.cc:2229] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:28:10.493595 31993 raft_consensus.cc:2258] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:10.493953 31994 ts_tablet_manager.cc:1905] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:10.496207 31993 ts_tablet_manager.cc:1905] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:10.498534 31992 tablet_replica.cc:330] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: stopping tablet replica
I20260806 20:28:10.499042 31992 raft_consensus.cc:2229] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:28:10.499806 31992 raft_consensus.cc:2258] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:10.501549 31994 ts_tablet_manager.cc:1918] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:10.501717 31994 log.cc:1196] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/7d84d30e57034775b189e98142fa7bab
I20260806 20:28:10.502305 31994 ts_tablet_manager.cc:1939] T 7d84d30e57034775b189e98142fa7bab P 4fbd7e39aa964ed7b5cbec2b24297cc3: Deleting consensus metadata
I20260806 20:28:10.504369 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.481348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49902 (local address 127.17.109.193:36313)
0806 20:28:10.481632 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.481638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.481659 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:28:10.481726 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.481732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.481814 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.481928 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:10.484724 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.486239 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.488521 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.489312 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.489468 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.493963 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:10.493990 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:10.494008 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:10.494119 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:10.498573 (+ 4454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.499971 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.499985 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.499988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.500418 (+ 430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.501021 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.501026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.501029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.501557 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:10.503317 (+ 1760us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.503481 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.503931 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260806 20:28:10.504420 31424 catalog_manager.cc:4841] TS 4fbd7e39aa964ed7b5cbec2b24297cc3 (127.17.109.194:37415): tablet 7d84d30e57034775b189e98142fa7bab (table test-table [id=a86854043eed48d0818cb8c07cf7799e]) successfully deleted
W20260806 20:28:10.505471 31424 catalog_manager.cc:4808] TS 4fbd7e39aa964ed7b5cbec2b24297cc3 (127.17.109.194:37415): delete failed for tablet 7d84d30e57034775b189e98142fa7bab because the tablet was not found. No further retry: Not found: Tablet not found: 7d84d30e57034775b189e98142fa7bab
I20260806 20:28:10.505959 31993 ts_tablet_manager.cc:1918] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:10.506021 31643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:10.506155 31993 log.cc:1196] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/7d84d30e57034775b189e98142fa7bab
I20260806 20:28:10.508225 31992 ts_tablet_manager.cc:1905] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:10.515424 31992 ts_tablet_manager.cc:1918] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:10.515672 31992 log.cc:1196] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/7d84d30e57034775b189e98142fa7bab
I20260806 20:28:10.516242 31992 ts_tablet_manager.cc:1939] T 7d84d30e57034775b189e98142fa7bab P 2f049a9ec0094bb6a700d1d6d6cd4e50: Deleting consensus metadata
I20260806 20:28:10.517905 31425 catalog_manager.cc:4841] TS 2f049a9ec0094bb6a700d1d6d6cd4e50 (127.17.109.195:42393): tablet 7d84d30e57034775b189e98142fa7bab (table test-table [id=a86854043eed48d0818cb8c07cf7799e]) successfully deleted
I20260806 20:28:10.636947 31993 ts_tablet_manager.cc:1939] T 7d84d30e57034775b189e98142fa7bab P 13082b2594a44613ba394394fef4b47c: Deleting consensus metadata
I20260806 20:28:10.637218 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 31479
W20260806 20:28:10.663153 31423 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.109.193:36313 (error 108)
I20260806 20:28:10.664191 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 31653
I20260806 20:28:10.665355 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.664728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49904)
0806 20:28:10.665017 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.665167 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
W20260806 20:28:10.665661 31423 catalog_manager.cc:4584] TS 13082b2594a44613ba394394fef4b47c (127.17.109.193:36313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7d84d30e57034775b189e98142fa7bab: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
I20260806 20:28:10.679840 31978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.664988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59934 (local address 127.17.109.194:37415)
0806 20:28:10.665276 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.665282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.665308 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:28:10.665475 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.665483 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.665576 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.665700 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:10.667896 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.668634 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.670060 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.670483 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.670625 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.673709 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:10.673723 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:10.673744 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:10.673776 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:10.676826 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.677581 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.677592 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.677601 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.677759 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.678151 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.678162 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.678164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.678532 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:10.678864 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.679163 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.679413 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 20:28:10.680611 31792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:10.719439 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.718849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49916)
0806 20:28:10.719110 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.719248 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 20:28:10.720741 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.720280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49930)
0806 20:28:10.720463 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.720564 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 20:28:10.774183 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 31653
I20260806 20:28:10.781564 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.780964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49938)
0806 20:28:10.781229 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.781349 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260806 20:28:10.782820 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.782370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49942)
0806 20:28:10.782557 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.782651 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 20:28:10.795266 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 31801
I20260806 20:28:10.812803 31963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.796164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.17.109.195:42393)
0806 20:28:10.796398 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:10.796403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:10.796421 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:28:10.796640 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:10.796644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:10.796715 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:10.796846 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:10.798954 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.799989 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.802071 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:10.802528 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:10.802631 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:10.806204 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:10.806214 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:10.806233 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:10.806272 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:10.809612 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.810578 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.810584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.810587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.810726 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:10.811190 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:10.811193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:10.811195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:10.811689 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:10.811991 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:10.812339 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 20:28:10.812510 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 20:28:10.813581 31931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:10.863749 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.863116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49950)
0806 20:28:10.863368 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.863496 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 20:28:10.864809 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:10.864419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49956)
0806 20:28:10.864620 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:10.864694 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 20:28:10.936254 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 31801
I20260806 20:28:10.959455 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 31406
I20260806 20:28:10.961066 31470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:11.077798 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:11.076897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36313 (local address 127.0.0.1:49958)
0806 20:28:11.077139 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:11.077279 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 20:28:11.078145 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 31406
I20260806 20:28:11.108914 17847 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 20:28:11.454927 17847 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 20:28:11.455137 17847 mini_ranger.cc:66] Stopping Ranger...
I20260806 20:28:11.800926 17847 mini_ranger.cc:68] Stopped Ranger
2026-08-06 20:28:11.801 UTC [30403] LOG: received smart shutdown request
2026-08-06 20:28:11.804 UTC [30403] LOG: background worker "logical replication launcher" (PID 30412) exited with exit code 1
2026-08-06 20:28:11.804 UTC [30407] LOG: shutting down
2026-08-06 20:28:11.990 UTC [30403] LOG: database system is shut down
2026-08-06T20:28:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55554 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:28:12 dist-test-slave-6k22 krb5kdc[32009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:28:12 dist-test-slave-6k22 krb5kdc[32009](info): set up 2 sockets
Aug 06 20:28:12 dist-test-slave-6k22 krb5kdc[32009](info): commencing operation
krb5kdc: starting...
W20260806 20:28:13.307804 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.003s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:28:13 dist-test-slave-6k22 krb5kdc[32009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:28:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:28:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:28:13.603185 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:45523
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:45523,127.17.109.253:35103,127.17.109.252:35235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:13.947103 32023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:13.947409 32023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:13.947510 32023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:13.955185 32023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:13.955307 32023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:13.955361 32023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:13.955428 32023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:13.955487 32023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:13.966156 32023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:45523,127.17.109.253:35103,127.17.109.252:35235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:45523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:13.967645 32023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:13.969326 32023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:13.983351 32028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:13.983281 32031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:13.983323 32029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:13.984437 32023 server_base.cc:1034] running on GCE node
I20260806 20:28:13.985327 32023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:13.986917 32023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:13.988204 32023 hybrid_clock.cc:648] HybridClock initialized: now 1786048093988130 us; error 63 us; skew 500 ppm
I20260806 20:28:13.988600 32023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:13 dist-test-slave-6k22 krb5kdc[32009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048093, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:13.997901 32023 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:28:13.999644 32023 webserver.cc:460] Webserver started at http://127.17.109.254:41615/ using document root <none> and password file <none>
I20260806 20:28:14.000357 32023 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:14.000463 32023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:14.000790 32023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:14.003943 32023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "73783d72f5164100b689f9d5b58cf5ae"
format_stamp: "Formatted at 2026-08-06 20:28:13 on dist-test-slave-6k22"
server_key: "66433db08b9a36cccfc97fdf83b6ed0a"
server_key_iv: "8378f84c9657a326dfd4e880fc13dee5"
server_key_version: "encryptionkey@0"
I20260806 20:28:14.004793 32023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "73783d72f5164100b689f9d5b58cf5ae"
format_stamp: "Formatted at 2026-08-06 20:28:13 on dist-test-slave-6k22"
server_key: "66433db08b9a36cccfc97fdf83b6ed0a"
server_key_iv: "8378f84c9657a326dfd4e880fc13dee5"
server_key_version: "encryptionkey@0"
I20260806 20:28:14.010239 32023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:28:14.013947 32038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:14.015332 32023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:28:14.015532 32023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "73783d72f5164100b689f9d5b58cf5ae"
format_stamp: "Formatted at 2026-08-06 20:28:13 on dist-test-slave-6k22"
server_key: "66433db08b9a36cccfc97fdf83b6ed0a"
server_key_iv: "8378f84c9657a326dfd4e880fc13dee5"
server_key_version: "encryptionkey@0"
I20260806 20:28:14.015779 32023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:14.029192 32023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:14.033643 32023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:14.033973 32023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:14.053874 32023 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:45523
I20260806 20:28:14.053911 32094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:45523 every 8 connection(s)
I20260806 20:28:14.055589 32023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:28:14.059479 32095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:14.062228 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 32023
I20260806 20:28:14.062475 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:28:14.063320 17847 external_mini_cluster.cc:1428] Setting key 4c69179aa1b01ce6e5e355f5a99cc720
I20260806 20:28:14.069420 32095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:28:14 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048093, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.109.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.253@KRBTEST.COM" created.
I20260806 20:28:14.097469 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.073926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.17.109.254:45523)
0806 20:28:14.076042 (+ 2116us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:14.076048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:14.076098 (+ 50us) server_negotiation.cc:407] Connection header received
0806 20:28:14.078022 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:14.078034 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:14.078379 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:14.078598 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:14.079907 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.081941 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.083512 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.084573 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.084802 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:14.088533 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:14.088547 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:14.088568 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:14.088625 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:14.092788 (+ 4163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:14.093568 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:14.093581 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:14.093587 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:14.093818 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:14.094211 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:14.094214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:14.094216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:14.094878 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:14.095044 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:14.095735 (+ 691us) server_negotiation.cc:299] Negotiation successful
0806 20:28:14.096052 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1915,"spinlock_wait_cycles":49024,"thread_start_us":1655,"threads_started":1}
I20260806 20:28:14.098021 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.073926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:45523 (local address 127.0.0.1:58862)
0806 20:28:14.074941 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.075002 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:14.077611 (+ 2609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:14.078634 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:14.078639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:14.079238 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:14.079689 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:14.079716 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.082105 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.082109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:14.083360 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:14.083373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.084826 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.084832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:14.084957 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:14.085689 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:14.085712 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:14.088267 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:14.092945 (+ 4678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:14.092955 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:14.092966 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:14.093399 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:14.093935 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:14.093939 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:14.093941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:14.094097 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:14.095007 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:14.095013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:14.095220 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 20:28:14.095516 (+ 296us) client_negotiation.cc:240] Negotiation successful
0806 20:28:14.096273 (+ 757us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":157,"thread_start_us":427,"threads_started":1}
I20260806 20:28:14.100859 32095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:28:14.101979 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.101601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:57946)
0806 20:28:14.101774 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.101880 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 20:28:14.102265 32042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
I20260806 20:28:14.103883 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.103490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:57960)
0806 20:28:14.103707 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.103773 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 20:28:14.104099 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:35103: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab.
Entry for principal kudu/127.17.109.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab.
I20260806 20:28:14.129320 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.253:35103
--webserver_interface=127.17.109.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab
--principal=kudu/127.17.109.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:45523,127.17.109.253:35103,127.17.109.252:35235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:28:14.153548 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } attempt: 1
I20260806 20:28:14.155063 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.154283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:57968)
0806 20:28:14.154590 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.154809 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 20:28:14.156492 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.155923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:57984)
0806 20:28:14.156132 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.156270 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 20:28:14.156744 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:35103: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
I20260806 20:28:14.225024 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } attempt: 2
I20260806 20:28:14.226414 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.225713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:57998)
0806 20:28:14.226046 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.226208 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 20:28:14.227728 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.227178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:58000)
0806 20:28:14.227351 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.227475 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 20:28:14.228008 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:35103: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
I20260806 20:28:14.319294 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } attempt: 3
I20260806 20:28:14.320789 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.320087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:58004)
0806 20:28:14.320386 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.320582 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 20:28:14.322189 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.321638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:58016)
0806 20:28:14.321854 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.321969 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 20:28:14.322436 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:35103: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
I20260806 20:28:14.465737 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } attempt: 4
I20260806 20:28:14.467275 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.466583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:58028)
0806 20:28:14.467004 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.467117 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
I20260806 20:28:14.468613 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.468151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:58036)
0806 20:28:14.468311 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.468407 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 20:28:14.469126 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:35103: Network error: Client connection negotiation failed: client connection to 127.17.109.253:35103: connect: Connection refused (error 111)
W20260806 20:28:14.489370 32103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:14.489734 32103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:14.489893 32103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:14.497440 32103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:14.497596 32103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:14.497699 32103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:14.497802 32103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:14.497884 32103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:14.508451 32103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:45523,127.17.109.253:35103,127.17.109.252:35235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab
--principal=kudu/127.17.109.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.253:35103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.109.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:14.509835 32103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:14.511180 32103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:14.523481 32109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:14.523520 32110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:14.523481 32112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:14.524394 32103 server_base.cc:1034] running on GCE node
I20260806 20:28:14.525279 32103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:14.526712 32103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:14.527968 32103 hybrid_clock.cc:648] HybridClock initialized: now 1786048094527907 us; error 75 us; skew 500 ppm
I20260806 20:28:14.528327 32103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:14 dist-test-slave-6k22 krb5kdc[32009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048094, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:14.531456 32103 init.cc:377] Logged in from keytab as kudu/127.17.109.253@KRBTEST.COM (short username kudu)
I20260806 20:28:14.532998 32103 webserver.cc:460] Webserver started at http://127.17.109.253:36647/ using document root <none> and password file <none>
I20260806 20:28:14.533649 32103 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:14.533751 32103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:14.534011 32103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:14.537057 32103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/data/instance:
uuid: "4dbcc35ba1474847b490da7b6e66ae67"
format_stamp: "Formatted at 2026-08-06 20:28:14 on dist-test-slave-6k22"
server_key: "930a62dde8ce34aba93e6102828293a1"
server_key_iv: "d08f6029920a92d4a9699265ee529c43"
server_key_version: "encryptionkey@0"
I20260806 20:28:14.537811 32103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/wal/instance:
uuid: "4dbcc35ba1474847b490da7b6e66ae67"
format_stamp: "Formatted at 2026-08-06 20:28:14 on dist-test-slave-6k22"
server_key: "930a62dde8ce34aba93e6102828293a1"
server_key_iv: "d08f6029920a92d4a9699265ee529c43"
server_key_version: "encryptionkey@0"
I20260806 20:28:14.543125 32103 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:28:14.546854 32119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:14.548321 32103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 20:28:14.548511 32103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/wal
uuid: "4dbcc35ba1474847b490da7b6e66ae67"
format_stamp: "Formatted at 2026-08-06 20:28:14 on dist-test-slave-6k22"
server_key: "930a62dde8ce34aba93e6102828293a1"
server_key_iv: "d08f6029920a92d4a9699265ee529c43"
server_key_version: "encryptionkey@0"
I20260806 20:28:14.548719 32103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:14.571384 32103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:14.575729 32103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:14.576048 32103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:14.595803 32103 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.253:35103
I20260806 20:28:14.595866 32172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.253:35103 every 8 connection(s)
I20260806 20:28:14.597224 32103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
I20260806 20:28:14.598657 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 32103
I20260806 20:28:14.598826 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-1/wal/instance
I20260806 20:28:14.599275 17847 external_mini_cluster.cc:1428] Setting key b92048f7c2e41e8183144b28a8a8b98b
I20260806 20:28:14.601768 32173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:14.608381 32173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:28:14 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048094, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.109.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.252@KRBTEST.COM" created.
I20260806 20:28:14.630613 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.610994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.17.109.254:45523)
0806 20:28:14.611656 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:14.611665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:14.612664 (+ 999us) server_negotiation.cc:407] Connection header received
0806 20:28:14.614562 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:14.614571 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:14.614666 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:14.614831 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:14.616379 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.617664 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.619244 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.620051 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.620206 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:14.624057 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:14.624068 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:14.624075 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:14.624138 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:14.627335 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:14.628163 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:14.628174 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:14.628178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:14.628303 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:14.628791 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:14.628795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:14.628798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:14.629088 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:14.629263 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:14.630000 (+ 737us) server_negotiation.cc:299] Negotiation successful
0806 20:28:14.630237 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":322,"threads_started":1}
I20260806 20:28:14.631661 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.611647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:45523 (local address 127.0.0.1:58870)
0806 20:28:14.612473 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.612519 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:14.614277 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:14.615009 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:14.615017 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:14.615551 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:14.616183 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:14.616200 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.617829 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.617833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:14.619056 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:14.619068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.620293 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.620301 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:14.620522 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:14.621314 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:14.621336 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:14.623821 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:14.627462 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:14.627468 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:14.627480 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:14.628027 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:14.628429 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:14.628434 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:14.628438 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:14.628651 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:14.629226 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:14.629234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:14.629614 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 20:28:14.629894 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 20:28:14.630406 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":367,"threads_started":1}
I20260806 20:28:14.633445 32173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:28:14 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048094, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
I20260806 20:28:14.650633 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.634445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58050 (local address 127.17.109.253:35103)
0806 20:28:14.635215 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:14.635225 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:14.635260 (+ 35us) server_negotiation.cc:407] Connection header received
0806 20:28:14.635322 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:14.635333 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:14.635603 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:14.635763 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:14.636429 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.638114 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.639953 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.640489 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.640589 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:14.644627 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:14.644640 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:14.644654 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:14.644689 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:14.647543 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:14.648448 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:14.648456 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:14.648458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:14.648588 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:14.649017 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:14.649020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:14.649022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:14.649363 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:14.649531 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:14.649980 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 20:28:14.650330 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":577,"thread_start_us":436,"threads_started":1}
I20260806 20:28:14.650722 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.634316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:58050)
0806 20:28:14.634538 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.634559 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:14.634673 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:14.635773 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:14.635777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:14.636138 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:14.636273 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:14.636281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.638252 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.638258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:14.639808 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:14.639826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.640654 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.640663 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:14.640919 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:14.641547 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:14.641574 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:14.644409 (+ 2835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:14.647755 (+ 3346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:14.647763 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:14.647770 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:14.648314 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:14.648699 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:14.648706 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:14.648712 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:14.648910 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:14.649521 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:14.649529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:14.649696 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 20:28:14.649855 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 20:28:14.650212 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 20:28:14.653435 32173 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:28:14.654613 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.654034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59858)
0806 20:28:14.654231 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.654387 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 20:28:14.654883 32123 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
I20260806 20:28:14.656502 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.656182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59862)
0806 20:28:14.656321 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.656403 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 20:28:14.656723 32173 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab.
Entry for principal kudu/127.17.109.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab.
I20260806 20:28:14.664191 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.252:35235
--webserver_interface=127.17.109.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab
--principal=kudu/127.17.109.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:45523,127.17.109.253:35103,127.17.109.252:35235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:28:14.706110 32173 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 1
I20260806 20:28:14.707454 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.706867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59870)
0806 20:28:14.707095 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.707260 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 20:28:14.708532 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.708131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59884)
0806 20:28:14.708302 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.708406 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260806 20:28:14.708761 32173 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
I20260806 20:28:14.768442 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } attempt: 5
Aug 06 20:28:14 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048093, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
I20260806 20:28:14.777033 32173 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 2
I20260806 20:28:14.778152 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.777646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59892)
0806 20:28:14.777858 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.777972 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 20:28:14.779825 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.779398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59908)
0806 20:28:14.779577 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.779658 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 20:28:14.780131 32173 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
I20260806 20:28:14.784129 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.769547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.17.109.253:35103)
0806 20:28:14.769781 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:14.769790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:14.769819 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:28:14.769917 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:14.769926 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:14.770027 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:14.770164 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:14.770956 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.772281 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.773533 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.774230 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.774363 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:14.778074 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:14.778081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:14.778087 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:14.778133 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:14.781440 (+ 3307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:14.782285 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:14.782290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:14.782292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:14.782414 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:14.782829 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:14.782832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:14.782834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:14.783120 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:14.783230 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:14.783772 (+ 542us) server_negotiation.cc:299] Negotiation successful
0806 20:28:14.783939 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 20:28:14.784343 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.769319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:58056)
0806 20:28:14.769562 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.769586 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:14.769726 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:14.770213 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:14.770216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:14.770609 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:14.770769 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:14.770779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.772433 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.772437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:14.773370 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:14.773383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:14.774377 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:14.774381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:14.774510 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:14.775187 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:14.775205 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:14.777859 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:14.781610 (+ 3751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:14.781619 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:14.781624 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:14.782147 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:14.782531 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:14.782536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:14.782540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:14.782722 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:14.783249 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:14.783254 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:14.783425 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 20:28:14.783612 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 20:28:14.783941 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 20:28:14.785526 32095 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:28:14.786616 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.786155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59924)
0806 20:28:14.786366 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.786467 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 20:28:14.787546 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.787217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59928)
0806 20:28:14.787367 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.787448 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 20:28:14.787859 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
I20260806 20:28:14.839107 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 1
I20260806 20:28:14.840629 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.839894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59942)
0806 20:28:14.840204 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.840371 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 20:28:14.841925 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.841422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59946)
0806 20:28:14.841623 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.841735 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 20:28:14.842231 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
I20260806 20:28:14.871410 32173 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 3
I20260806 20:28:14.872727 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.872226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59962)
0806 20:28:14.872476 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.872566 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 20:28:14.873819 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.873392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59970)
0806 20:28:14.873627 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.873701 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260806 20:28:14.874040 32173 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
I20260806 20:28:14.910512 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 2
I20260806 20:28:14.911859 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.911219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59976)
0806 20:28:14.911495 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.911659 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 20:28:14.913093 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:14.912642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59982)
0806 20:28:14.912844 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:14.912943 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 20:28:14.913358 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
I20260806 20:28:15.017254 32173 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 4
I20260806 20:28:15.018440 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.017909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:59994)
0806 20:28:15.018155 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.018252 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 20:28:15.019642 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.019233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:60002)
0806 20:28:15.019384 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.019464 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 20:28:15.019629 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 3
W20260806 20:28:15.019951 32173 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
I20260806 20:28:15.020635 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.020209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:60004)
0806 20:28:15.020412 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.020493 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260806 20:28:15.021109 32181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:15.021448 32181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:15.021605 32181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 20:28:15.021719 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.021299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:60008)
0806 20:28:15.021477 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.021580 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 20:28:15.021965 32095 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35235: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35235: connect: Connection refused (error 111)
W20260806 20:28:15.029474 32181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:15.029623 32181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:15.029723 32181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:15.029834 32181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:15.029911 32181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:15.040104 32181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:45523,127.17.109.253:35103,127.17.109.252:35235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab
--principal=kudu/127.17.109.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.252:35235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.109.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:15.041556 32181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:15.042871 32181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:15.055439 32188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:15.055778 32186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:15.056525 32190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:15.056849 32181 server_base.cc:1034] running on GCE node
I20260806 20:28:15.057744 32181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:15.059547 32181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:15.060813 32181 hybrid_clock.cc:648] HybridClock initialized: now 1786048095060768 us; error 35 us; skew 500 ppm
I20260806 20:28:15.061254 32181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048095, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:15.065492 32181 init.cc:377] Logged in from keytab as kudu/127.17.109.252@KRBTEST.COM (short username kudu)
I20260806 20:28:15.067631 32181 webserver.cc:460] Webserver started at http://127.17.109.252:42619/ using document root <none> and password file <none>
I20260806 20:28:15.068620 32181 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:15.068765 32181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:15.069166 32181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:15.073704 32181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/data/instance:
uuid: "bc66cf6538c34c6b834c2cf4f9b67203"
format_stamp: "Formatted at 2026-08-06 20:28:15 on dist-test-slave-6k22"
server_key: "feb091e30417bf5a65257e9c885c997b"
server_key_iv: "e2e136d924ea6095e9d30bc6425e1973"
server_key_version: "encryptionkey@0"
I20260806 20:28:15.074755 32181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/wal/instance:
uuid: "bc66cf6538c34c6b834c2cf4f9b67203"
format_stamp: "Formatted at 2026-08-06 20:28:15 on dist-test-slave-6k22"
server_key: "feb091e30417bf5a65257e9c885c997b"
server_key_iv: "e2e136d924ea6095e9d30bc6425e1973"
server_key_version: "encryptionkey@0"
I20260806 20:28:15.082590 32181 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260806 20:28:15.087674 32197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:15.089781 32181 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260806 20:28:15.089994 32181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/wal
uuid: "bc66cf6538c34c6b834c2cf4f9b67203"
format_stamp: "Formatted at 2026-08-06 20:28:15 on dist-test-slave-6k22"
server_key: "feb091e30417bf5a65257e9c885c997b"
server_key_iv: "e2e136d924ea6095e9d30bc6425e1973"
server_key_version: "encryptionkey@0"
I20260806 20:28:15.090253 32181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:15.125763 32181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:15.130183 32181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:15.130419 32181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:15.149474 32181 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.252:35235
I20260806 20:28:15.149507 32250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.252:35235 every 8 connection(s)
I20260806 20:28:15.150851 32181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
I20260806 20:28:15.153692 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 32181
I20260806 20:28:15.153894 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786047878953513-17847-0/minicluster-data/master-2/wal/instance
I20260806 20:28:15.154419 17847 external_mini_cluster.cc:1428] Setting key d49abbc92e3d95704f0f54b6a276b351
I20260806 20:28:15.155287 32251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:15.166106 32251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048095, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:15.179282 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.156440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60020 (local address 127.17.109.252:35235)
0806 20:28:15.157342 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.157362 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.157417 (+ 55us) server_negotiation.cc:407] Connection header received
0806 20:28:15.159241 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.159272 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.159830 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.160309 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.161721 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.163880 (+ 2159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.165188 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.166259 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.166465 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.171437 (+ 4972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.171448 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.171458 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.171493 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.174638 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.175381 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.175390 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.175403 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.175627 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.176061 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.176066 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.176069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.176590 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.176898 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.177397 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.177850 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":365,"threads_started":1}
I20260806 20:28:15.185035 32255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.168971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:45523 (local address 127.0.0.1:58872)
0806 20:28:15.169854 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.169912 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:15.170114 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:15.170777 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:15.170785 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:15.171378 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:15.171764 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.171775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.173087 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.173091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.174205 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.174217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.174963 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.174969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.175140 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.175749 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:15.175770 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:15.177934 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:15.181815 (+ 3881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.181830 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.181840 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.182410 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.182981 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.182986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.182990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.183165 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.184029 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:15.184037 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:15.184236 (+ 199us) client_negotiation.cc:769] Sending connection context
0806 20:28:15.184440 (+ 204us) client_negotiation.cc:240] Negotiation successful
0806 20:28:15.184770 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":377,"threads_started":1}
I20260806 20:28:15.186192 32256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.169031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.17.109.254:45523)
0806 20:28:15.170425 (+ 1394us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.170431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.170458 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:28:15.170509 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.170513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.170580 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.170723 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.171922 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.172913 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.174351 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.174777 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.174936 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.178131 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.178138 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.178142 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.178182 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.181636 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.182566 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.182571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.182574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.182858 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.183325 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.183335 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.183339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.183876 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.184056 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.185570 (+ 1514us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.185839 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1251,"thread_start_us":1111,"threads_started":1}
I20260806 20:28:15.187611 32251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048095, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
I20260806 20:28:15.199184 32095 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 4
I20260806 20:28:15.201645 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.182794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.17.109.254:45523)
0806 20:28:15.184147 (+ 1353us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.184153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.184183 (+ 30us) server_negotiation.cc:407] Connection header received
0806 20:28:15.184230 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.184260 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.184332 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.184446 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.185854 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.186964 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.188605 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.189320 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.189414 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.194769 (+ 5355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.194775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.194778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.194810 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.198168 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.199158 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.199166 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.199168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.199311 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.199915 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.199920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.199922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.200260 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.200486 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.200901 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.201161 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":939,"threads_started":1}
I20260806 20:28:15.203516 32255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.188547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:35103 (local address 127.0.0.1:58072)
0806 20:28:15.188691 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.188711 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:15.188810 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:15.189130 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:15.189133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:15.189523 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:15.189721 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.189732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.190959 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.190967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.192523 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.192539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.193460 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.193464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.193690 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.194239 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:15.194256 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:15.196515 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:15.200586 (+ 4071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.200589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.200592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.201129 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.201575 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.201579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.201582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.201768 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.202475 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:15.202481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:15.202671 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 20:28:15.202857 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 20:28:15.203174 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 20:28:15.204794 32178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.188446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.17.109.253:35103)
0806 20:28:15.188632 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.188637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.188797 (+ 160us) server_negotiation.cc:407] Connection header received
0806 20:28:15.188914 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.188923 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.189006 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.189108 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.190003 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.190801 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.192673 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.193297 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.193432 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.196767 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.196775 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.196779 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.196823 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.200444 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.201289 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.201295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.201300 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.201467 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.201967 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.201972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.201976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.202336 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.202489 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.202964 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.204303 (+ 1339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 20:28:15.206053 32251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048093, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048095, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
I20260806 20:28:15.217018 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.199849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:60024)
0806 20:28:15.200041 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.200068 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:15.200149 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:15.201019 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:15.201025 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:15.201547 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:15.201800 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.201815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.203245 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.203251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.204493 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.204506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.205282 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.205287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.205482 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.206037 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:15.206079 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:15.208632 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:15.212544 (+ 3912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.212552 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.212558 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.213098 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.215027 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.215033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.215037 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.215217 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.215885 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:15.215890 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:15.216056 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 20:28:15.216279 (+ 223us) client_negotiation.cc:240] Negotiation successful
0806 20:28:15.216600 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 20:28:15.217749 32254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.199916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.17.109.252:35235)
0806 20:28:15.200702 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.200708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.200727 (+ 19us) server_negotiation.cc:407] Connection header received
0806 20:28:15.200786 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.200790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.200862 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.201972 (+ 1110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.202117 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.203097 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.204635 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.205129 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.205263 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.208808 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.208813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.208818 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.208856 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.212394 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.213239 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.213246 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.213251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.213393 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.215368 (+ 1975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.215375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.215379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.215757 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.215919 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.217189 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.217449 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658}
I20260806 20:28:15.222975 32255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.206696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:60028)
0806 20:28:15.206871 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.206889 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:15.207011 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:15.208109 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:15.208114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:15.208600 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:15.208780 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.208793 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.210457 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.210462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.211977 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.211995 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.212884 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.212890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.213081 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.213657 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:15.213673 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:15.216170 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:15.220380 (+ 4210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.220384 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.220388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.220868 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.221274 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.221278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.221282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.221437 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.222032 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:15.222037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:15.222181 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 20:28:15.222338 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 20:28:15.222630 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 20:28:15.223176 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.203704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58078 (local address 127.17.109.253:35103)
0806 20:28:15.204325 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.204330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.204355 (+ 25us) server_negotiation.cc:407] Connection header received
0806 20:28:15.204399 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.204405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.204477 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.204619 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.206585 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.207802 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.209475 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.210084 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.210220 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.215615 (+ 5395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.215623 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.215629 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.215669 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.219653 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.220562 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.220569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.220572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.220713 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.221182 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.221187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.221192 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.221523 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.221791 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.222322 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.222546 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":313,"threads_started":1}
I20260806 20:28:15.223984 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.207086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.17.109.252:35235)
0806 20:28:15.207814 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.207821 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.207844 (+ 23us) server_negotiation.cc:407] Connection header received
0806 20:28:15.207887 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.207892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.207955 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.208067 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.208960 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.210334 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.212151 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.212729 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.212849 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.216388 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.216394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.216399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.216432 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.220247 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.220995 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.221000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.221003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.221148 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.221562 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.221570 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.221576 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.221903 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.222044 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.223368 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.223600 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":367,"threads_started":1}
I20260806 20:28:15.224033 32095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae: Bootstrap starting.
I20260806 20:28:15.227996 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:28:15.228333 32095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:15.229483 32251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: Bootstrap starting.
I20260806 20:28:15.229679 32095 log.cc:824] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:15.233189 32095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae: No bootstrap required, opened a new log
I20260806 20:28:15.233449 32251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:15.235232 32251 log.cc:824] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:15.244838 32251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: No bootstrap required, opened a new log
I20260806 20:28:15.246110 32095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
I20260806 20:28:15.246546 32095 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:15.246713 32095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73783d72f5164100b689f9d5b58cf5ae, State: Initialized, Role: FOLLOWER
I20260806 20:28:15.249332 32095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
I20260806 20:28:15.256652 32251 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
I20260806 20:28:15.257140 32251 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:15.257311 32251 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc66cf6538c34c6b834c2cf4f9b67203, State: Initialized, Role: FOLLOWER
I20260806 20:28:15.258301 32251 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
I20260806 20:28:15.261831 32095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:15.265466 32271 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.265913 32271 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:28:15.266914 32269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.267333 32269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:28:15.267841 32251 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:15.269411 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.233680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.17.109.254:45523)
0806 20:28:15.233925 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.233933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.247837 (+ 13904us) server_negotiation.cc:407] Connection header received
0806 20:28:15.248048 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.248057 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.248172 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.248295 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.251407 (+ 3112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.252717 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.255700 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.256633 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.256772 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.261494 (+ 4722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.261501 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.261505 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.261548 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.265403 (+ 3855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.266593 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.266598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.266601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.266775 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.267395 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.267401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.267405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.267827 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.268096 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.268810 (+ 714us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.269055 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 20:28:15.278646 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.233759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.17.109.253:35103)
0806 20:28:15.233968 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.233978 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.250523 (+ 16545us) server_negotiation.cc:407] Connection header received
0806 20:28:15.250607 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.250615 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.250707 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.250811 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.253945 (+ 3134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.255034 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.263645 (+ 8611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.264577 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.264708 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.270321 (+ 5613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.270332 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.270340 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.270398 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.274694 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.275854 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.275862 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.275866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.276039 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.276855 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.276862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.276867 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.277308 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.277562 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.278030 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.278267 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 20:28:15.280514 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.251207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.17.109.252:35235)
0806 20:28:15.251411 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.251417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.258548 (+ 7131us) server_negotiation.cc:407] Connection header received
0806 20:28:15.258994 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.259000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.259110 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.259200 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.261306 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.262373 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.264381 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.265076 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.265213 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.269013 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.269020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.269024 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.269063 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.273533 (+ 4470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.275521 (+ 1988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.275526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.275529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.275708 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.278408 (+ 2700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.278414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.278418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.278809 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.279054 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.279940 (+ 886us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.280177 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 20:28:15.285300 32294 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260806 20:28:15.286983 32044 tablet.cc:2369] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:28:15.292184 32295 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 20:28:15.292316 32295 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 20:28:15.290210 32023 master_runner.cc:186] Error getting master registration for 127.17.109.253:35103: OK, instance_id { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" instance_seqno: 1786048094590268 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 20:28:15.300727 32023 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 20:28:15.292076 32181 master_runner.cc:186] Error getting master registration for 127.17.109.253:35103: OK, instance_id { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" instance_seqno: 1786048094590268 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 20:28:15.302232 32181 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 20:28:15.319192 32173 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } attempt: 5
Aug 06 20:28:15 dist-test-slave-6k22 krb5kdc[32009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048094, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
I20260806 20:28:15.335130 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.319866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35235 (local address 127.0.0.1:60040)
0806 20:28:15.320075 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:15.320103 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:15.320191 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:15.320588 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:15.320593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:15.321029 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:15.321188 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.321196 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.322667 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.322671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.323725 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:15.323762 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.324787 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.324792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:15.324971 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.325617 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:15.325635 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:15.328651 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:15.332817 (+ 4166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.332821 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.332824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.333178 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.333585 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:15.333589 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:15.333591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:15.333771 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:15.334372 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:15.334375 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:15.334503 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 20:28:15.334619 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 20:28:15.334855 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260806 20:28:15.335407 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:15.319992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60040 (local address 127.17.109.252:35235)
0806 20:28:15.320166 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:15.320177 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:15.320199 (+ 22us) server_negotiation.cc:407] Connection header received
0806 20:28:15.320304 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:15.320310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:15.320384 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:15.320505 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:15.321334 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.322536 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.323891 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:15.324583 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:15.324733 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:15.328838 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:15.328845 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:15.328853 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:15.328897 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:15.332673 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.333301 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.333307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.333310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.333475 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:15.333878 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:15.333883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:15.333886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:15.334259 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:15.334456 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:15.334772 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 20:28:15.335008 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 20:28:15.340705 32173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67: Bootstrap starting.
I20260806 20:28:15.343839 32173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:15.344871 32173 log.cc:824] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:15.347692 32173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67: No bootstrap required, opened a new log
I20260806 20:28:15.356339 32173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
I20260806 20:28:15.356683 32173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:15.356813 32173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbcc35ba1474847b490da7b6e66ae67, State: Initialized, Role: FOLLOWER
I20260806 20:28:15.357751 32173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
I20260806 20:28:15.358287 32299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:15.358727 32299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
I20260806 20:28:15.359723 32300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.360257 32300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:28:15.360546 32173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:15.360808 32299 leader_election.cc:288] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73783d72f5164100b689f9d5b58cf5ae (127.17.109.254:45523), bc66cf6538c34c6b834c2cf4f9b67203 (127.17.109.252:35235)
I20260806 20:28:15.361505 32224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dbcc35ba1474847b490da7b6e66ae67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" is_pre_election: true
I20260806 20:28:15.361902 32067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dbcc35ba1474847b490da7b6e66ae67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73783d72f5164100b689f9d5b58cf5ae" is_pre_election: true
I20260806 20:28:15.362161 32224 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbcc35ba1474847b490da7b6e66ae67 in term 0.
I20260806 20:28:15.362586 32067 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dbcc35ba1474847b490da7b6e66ae67 in term 0.
I20260806 20:28:15.362886 32123 leader_election.cc:302] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbcc35ba1474847b490da7b6e66ae67, bc66cf6538c34c6b834c2cf4f9b67203; no voters:
I20260806 20:28:15.363276 32299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:28:15.363412 32299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:28:15.363499 32299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:15.367969 32299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
I20260806 20:28:15.369109 32224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dbcc35ba1474847b490da7b6e66ae67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc66cf6538c34c6b834c2cf4f9b67203"
I20260806 20:28:15.369230 32067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4dbcc35ba1474847b490da7b6e66ae67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73783d72f5164100b689f9d5b58cf5ae"
I20260806 20:28:15.369329 32224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:15.369495 32067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:15.372784 32224 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbcc35ba1474847b490da7b6e66ae67 in term 1.
I20260806 20:28:15.373238 32123 leader_election.cc:302] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbcc35ba1474847b490da7b6e66ae67, bc66cf6538c34c6b834c2cf4f9b67203; no voters:
I20260806 20:28:15.373548 32300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:15.373930 32067 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dbcc35ba1474847b490da7b6e66ae67 in term 1.
I20260806 20:28:15.375666 32299 leader_election.cc:288] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [CANDIDATE]: Term 1 election: Requested vote from peers 73783d72f5164100b689f9d5b58cf5ae (127.17.109.254:45523), bc66cf6538c34c6b834c2cf4f9b67203 (127.17.109.252:35235)
I20260806 20:28:15.379837 32300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [term 1 LEADER]: Becoming Leader. State: Replica: 4dbcc35ba1474847b490da7b6e66ae67, State: Running, Role: LEADER
I20260806 20:28:15.380436 32300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } }
W20260806 20:28:15.382766 32203 tablet.cc:2369] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:28:15.386093 32299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dbcc35ba1474847b490da7b6e66ae67. Latest consensus state: current_term: 1 leader_uuid: "4dbcc35ba1474847b490da7b6e66ae67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.386439 32299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: This master's current role is: LEADER
W20260806 20:28:15.392002 32315 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 20:28:15.392205 32315 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 20:28:15.392515 32316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:28:15.394994 32316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:28:15.404901 32224 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [term 1 FOLLOWER]: Refusing update from remote peer 4dbcc35ba1474847b490da7b6e66ae67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:15.404896 32067 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [term 1 FOLLOWER]: Refusing update from remote peer 4dbcc35ba1474847b490da7b6e66ae67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:15.405930 32299 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:15.406303 32300 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:15.429009 32271 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dbcc35ba1474847b490da7b6e66ae67. Latest consensus state: current_term: 1 leader_uuid: "4dbcc35ba1474847b490da7b6e66ae67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.429505 32271 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:28:15.439931 32320 mvcc.cc:205] Tried to move back new op lower bound from 7315652998768115712 to 7315652998684295168. Current Snapshot: MvccSnapshot[applied={T|T < 7315652998768115712}]
I20260806 20:28:15.445436 32269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dbcc35ba1474847b490da7b6e66ae67. Latest consensus state: current_term: 1 leader_uuid: "4dbcc35ba1474847b490da7b6e66ae67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.445823 32269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:28:15.453081 32319 mvcc.cc:205] Tried to move back new op lower bound from 7315652998768115712 to 7315652998684295168. Current Snapshot: MvccSnapshot[applied={T|T < 7315652998768115712}]
I20260806 20:28:15.456327 32300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4dbcc35ba1474847b490da7b6e66ae67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.456718 32300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:15.457079 32300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4dbcc35ba1474847b490da7b6e66ae67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.457362 32300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67 [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:15.457984 32271 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4dbcc35ba1474847b490da7b6e66ae67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.458266 32271 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:28:15.459506 32316 catalog_manager.cc:1347] Generated new cluster ID: 0a039157bea14a3dabe9575ff709eebf
I20260806 20:28:15.459633 32316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:28:15.465229 32269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4dbcc35ba1474847b490da7b6e66ae67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73783d72f5164100b689f9d5b58cf5ae" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45523 } } peers { permanent_uuid: "4dbcc35ba1474847b490da7b6e66ae67" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 35103 } } peers { permanent_uuid: "bc66cf6538c34c6b834c2cf4f9b67203" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35235 } } }
I20260806 20:28:15.465507 32269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:28:15.482759 32316 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:28:15.484705 32316 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:28:15.498909 32316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dbcc35ba1474847b490da7b6e66ae67: Generated new TSK 0
I20260806 20:28:15.500166 32316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:28:16.290581 32294 catalog_manager.cc:1259] Loaded cluster ID: 0a039157bea14a3dabe9575ff709eebf
I20260806 20:28:16.290753 32294 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae: loading cluster ID for follower catalog manager: success
I20260806 20:28:16.294286 32294 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae: acquiring CA information for follower catalog manager: success
I20260806 20:28:16.294548 32295 catalog_manager.cc:1259] Loaded cluster ID: 0a039157bea14a3dabe9575ff709eebf
I20260806 20:28:16.294652 32295 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: loading cluster ID for follower catalog manager: success
I20260806 20:28:16.297094 32294 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73783d72f5164100b689f9d5b58cf5ae: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 20:28:16.297989 32295 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: acquiring CA information for follower catalog manager: success
I20260806 20:28:16.299819 32295 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc66cf6538c34c6b834c2cf4f9b67203: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 20:28:16.582239 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 32023
I20260806 20:28:16.583549 32089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:16.705272 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 32023
I20260806 20:28:16.729946 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 32103
I20260806 20:28:16.731192 32167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:16.843482 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 32103
I20260806 20:28:16.867141 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 32181
I20260806 20:28:16.868461 32244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:16.977600 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 32181
2026-08-06T20:28:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4839 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:28:17 dist-test-slave-6k22 krb5kdc[32335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:28:17 dist-test-slave-6k22 krb5kdc[32335](info): set up 2 sockets
Aug 06 20:28:17 dist-test-slave-6k22 krb5kdc[32335](info): commencing operation
krb5kdc: starting...
W20260806 20:28:18.152055 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.006s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:28:18 dist-test-slave-6k22 krb5kdc[32335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:28:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:28:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:28:18.451647 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46039
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:44665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:18.808199 32348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:18.808511 32348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:18.808610 32348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:18.816147 32348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:18.816293 32348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:18.816349 32348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:18.816413 32348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:18.816468 32348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:18.826861 32348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:44665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:18.828351 32348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:18.829705 32348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:18.843462 32353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:18.843466 32354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:18.843489 32356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:18.844560 32348 server_base.cc:1034] running on GCE node
I20260806 20:28:18.845436 32348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:18.846987 32348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:18.848270 32348 hybrid_clock.cc:648] HybridClock initialized: now 1786048098848207 us; error 46 us; skew 500 ppm
I20260806 20:28:18.848718 32348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:18 dist-test-slave-6k22 krb5kdc[32335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048098, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:18.852885 32348 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:28:18.854493 32348 webserver.cc:460] Webserver started at http://127.17.109.254:33925/ using document root <none> and password file <none>
I20260806 20:28:18.855227 32348 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:18.855340 32348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:18.855752 32348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:18.859069 32348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "92dc13b3a88d4612a579640917b88f47"
format_stamp: "Formatted at 2026-08-06 20:28:18 on dist-test-slave-6k22"
server_key: "362941605b4a95ce26c0e9e81f10e274"
server_key_iv: "97eacdfcc280642bfde2b747ccc4d133"
server_key_version: "encryptionkey@0"
I20260806 20:28:18.859917 32348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "92dc13b3a88d4612a579640917b88f47"
format_stamp: "Formatted at 2026-08-06 20:28:18 on dist-test-slave-6k22"
server_key: "362941605b4a95ce26c0e9e81f10e274"
server_key_iv: "97eacdfcc280642bfde2b747ccc4d133"
server_key_version: "encryptionkey@0"
I20260806 20:28:18.865505 32348 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:28:18.869230 32363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:18.870767 32348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:28:18.870977 32348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "92dc13b3a88d4612a579640917b88f47"
format_stamp: "Formatted at 2026-08-06 20:28:18 on dist-test-slave-6k22"
server_key: "362941605b4a95ce26c0e9e81f10e274"
server_key_iv: "97eacdfcc280642bfde2b747ccc4d133"
server_key_version: "encryptionkey@0"
I20260806 20:28:18.871196 32348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:18.890583 32348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:18.895529 32348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:18.895944 32348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:18.916747 32348 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:46039
I20260806 20:28:18.916792 32416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:46039 every 8 connection(s)
I20260806 20:28:18.918275 32348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:28:18.922120 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 32348
I20260806 20:28:18.922389 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:28:18.922442 32417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:18.923249 17847 external_mini_cluster.cc:1428] Setting key 1c036b4a7160bfe40ceac3c2353ac85e
I20260806 20:28:18.939029 32417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47: Bootstrap starting.
Aug 06 20:28:18 dist-test-slave-6k22 krb5kdc[32335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:18.943593 32417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:18.945149 32417 log.cc:824] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:18.949009 32417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47: No bootstrap required, opened a new log
I20260806 20:28:18.950649 32420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:18.925348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.17.109.254:46039)
0806 20:28:18.926321 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:18.926342 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:18.926398 (+ 56us) server_negotiation.cc:407] Connection header received
0806 20:28:18.928496 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:18.928535 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:18.929118 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:18.929628 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:18.931162 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:18.933320 (+ 2158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:18.934770 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:18.936031 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:18.936283 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:18.940798 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:18.940810 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:18.940840 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:18.940918 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:18.945361 (+ 4443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:18.946370 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:18.946378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:18.946384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:18.946694 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:18.947160 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:18.947165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:18.947168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:18.947927 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:18.948272 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:18.948727 (+ 455us) server_negotiation.cc:299] Negotiation successful
0806 20:28:18.949204 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":408,"threads_started":1}
I20260806 20:28:18.955065 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:28:18.958819 32417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dc13b3a88d4612a579640917b88f47" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46039 } }
I20260806 20:28:18.959249 32417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:18.959384 32417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92dc13b3a88d4612a579640917b88f47, State: Initialized, Role: FOLLOWER
I20260806 20:28:18.960085 32417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dc13b3a88d4612a579640917b88f47" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46039 } }
I20260806 20:28:18.960350 32417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:28:18.960533 32417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:28:18.960774 32417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:18.963166 32417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dc13b3a88d4612a579640917b88f47" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46039 } }
I20260806 20:28:18.963851 32417 leader_election.cc:302] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92dc13b3a88d4612a579640917b88f47; no voters:
I20260806 20:28:18.964505 32417 leader_election.cc:288] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:28:18.964695 32423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:18.965883 32423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [term 1 LEADER]: Becoming Leader. State: Replica: 92dc13b3a88d4612a579640917b88f47, State: Running, Role: LEADER
I20260806 20:28:18.966451 32423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dc13b3a88d4612a579640917b88f47" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46039 } }
I20260806 20:28:18.966883 32417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:18.969890 32424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92dc13b3a88d4612a579640917b88f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dc13b3a88d4612a579640917b88f47" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46039 } } }
I20260806 20:28:18.969957 32425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92dc13b3a88d4612a579640917b88f47. Latest consensus state: current_term: 1 leader_uuid: "92dc13b3a88d4612a579640917b88f47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92dc13b3a88d4612a579640917b88f47" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46039 } } }
I20260806 20:28:18.970304 32424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:18.970400 32425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47 [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:18.972528 32429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:28:18.979769 32429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:28:18.990689 32429 catalog_manager.cc:1347] Generated new cluster ID: 2efd517f0a54475f9598e07c41079577
I20260806 20:28:18.990875 32429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:28:19.016868 32429 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:28:19.019032 32429 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:28:19.035921 32429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92dc13b3a88d4612a579640917b88f47: Generated new TSK 0
I20260806 20:28:19.037186 32429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:28:19.067845 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 32348
I20260806 20:28:19.069166 32412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:19.166056 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 32348
2026-08-06T20:28:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2183 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:28:19 dist-test-slave-6k22 krb5kdc[32449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:28:19 dist-test-slave-6k22 krb5kdc[32449](info): set up 2 sockets
Aug 06 20:28:19 dist-test-slave-6k22 krb5kdc[32449](info): commencing operation
krb5kdc: starting...
W20260806 20:28:20.344092 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:28:20 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:28:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:28:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:28:20.648072 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34141
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:20.989419 32463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:20.989737 32463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:20.989831 32463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:20.997263 32463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:20.997380 32463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:20.997428 32463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:20.997488 32463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:20.997532 32463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:21.009343 32463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:21.010807 32463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:21.012390 32463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:21.025876 32469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:21.025949 32468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:21.025923 32471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:21.026887 32463 server_base.cc:1034] running on GCE node
I20260806 20:28:21.027882 32463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:21.029488 32463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:21.030754 32463 hybrid_clock.cc:648] HybridClock initialized: now 1786048101030700 us; error 35 us; skew 500 ppm
I20260806 20:28:21.031149 32463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:21 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048101, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:21.034559 32463 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:28:21.036330 32463 webserver.cc:460] Webserver started at http://127.17.109.254:43353/ using document root <none> and password file <none>
I20260806 20:28:21.037065 32463 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:21.037240 32463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:21.037595 32463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:21.041036 32463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "d4f8112ed6524564b85abd04865e4f3f"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "7d4ddf9d8bed820339fd56f71017d557"
server_key_iv: "a1bc55c373dd1adc2921a2d69caf2257"
server_key_version: "encryptionkey@0"
I20260806 20:28:21.041898 32463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "d4f8112ed6524564b85abd04865e4f3f"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "7d4ddf9d8bed820339fd56f71017d557"
server_key_iv: "a1bc55c373dd1adc2921a2d69caf2257"
server_key_version: "encryptionkey@0"
I20260806 20:28:21.047530 32463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:28:21.051363 32478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:21.052816 32463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 20:28:21.053027 32463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "d4f8112ed6524564b85abd04865e4f3f"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "7d4ddf9d8bed820339fd56f71017d557"
server_key_iv: "a1bc55c373dd1adc2921a2d69caf2257"
server_key_version: "encryptionkey@0"
I20260806 20:28:21.053261 32463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:21.075438 32463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:21.080294 32463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:21.080621 32463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:21.100608 32463 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:34141
I20260806 20:28:21.100646 32530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:34141 every 8 connection(s)
I20260806 20:28:21.102036 32463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:28:21.105924 32531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:21.106655 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 32463
I20260806 20:28:21.106901 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:28:21.107612 17847 external_mini_cluster.cc:1428] Setting key 5767f5b7a1c7a82913d77cdd3a3dff7d
I20260806 20:28:21.120776 32531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Bootstrap starting.
Aug 06 20:28:21 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:21.124185 32531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:21.125593 32531 log.cc:824] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:21.129205 32531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: No bootstrap required, opened a new log
I20260806 20:28:21.132897 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:21.109866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.17.109.254:34141)
0806 20:28:21.110727 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:21.110743 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:21.110797 (+ 54us) server_negotiation.cc:407] Connection header received
0806 20:28:21.112750 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:21.112779 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:21.113323 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:21.113802 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:21.115262 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:21.117369 (+ 2107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:21.118704 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:21.119802 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:21.120009 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:21.124438 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:21.124449 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:21.124468 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:21.124529 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:21.128657 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:21.129447 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:21.129454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:21.129460 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:21.129634 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:21.130074 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:21.130079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:21.130083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:21.130606 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:21.131002 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:21.131431 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 20:28:21.131894 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":371,"threads_started":1}
I20260806 20:28:21.137279 32531 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:21.137647 32531 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:21.137763 32531 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f8112ed6524564b85abd04865e4f3f, State: Initialized, Role: FOLLOWER
I20260806 20:28:21.138391 32531 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:21.138638 32531 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:28:21.138784 32531 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:28:21.138958 32531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:21.141548 32531 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:21.142019 32531 leader_election.cc:302] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4f8112ed6524564b85abd04865e4f3f; no voters:
I20260806 20:28:21.142572 32531 leader_election.cc:288] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:28:21.142688 32537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:21.143882 32537 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 1 LEADER]: Becoming Leader. State: Replica: d4f8112ed6524564b85abd04865e4f3f, State: Running, Role: LEADER
I20260806 20:28:21.144455 32537 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:21.144847 32531 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:21.147516 32539 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4f8112ed6524564b85abd04865e4f3f. Latest consensus state: current_term: 1 leader_uuid: "d4f8112ed6524564b85abd04865e4f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } } }
I20260806 20:28:21.147940 32539 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:21.147483 32538 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4f8112ed6524564b85abd04865e4f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } } }
I20260806 20:28:21.148590 32538 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:21.149828 32545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:28:21.154255 32545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:28:21.163635 32545 catalog_manager.cc:1347] Generated new cluster ID: 6bea52ace37d4f508c40c163459470b3
I20260806 20:28:21.163753 32545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:28:21.176920 32545 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:28:21.178805 32545 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:28:21.186448 32545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Generated new TSK 0
I20260806 20:28:21.187485 32545 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:28:21.263238 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34141
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:21.618546 32559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:21.618848 32559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:21.619050 32559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:21.626979 32559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:21.627146 32559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:21.627357 32559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:28:21.640584 32559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:21.641973 32559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:21.643410 32559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:21.657960 32565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:21.658003 32564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:21.658653 32567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:21.659299 32559 server_base.cc:1034] running on GCE node
I20260806 20:28:21.660076 32559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:21.661113 32559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:21.662371 32559 hybrid_clock.cc:648] HybridClock initialized: now 1786048101662338 us; error 54 us; skew 500 ppm
I20260806 20:28:21.662752 32559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:21 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048101, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:21.666198 32559 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:28:21.667969 32559 webserver.cc:460] Webserver started at http://127.17.109.193:41751/ using document root <none> and password file <none>
I20260806 20:28:21.668737 32559 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:21.668902 32559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:21.669250 32559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:21.672574 32559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "65031948d14d47838c64d64d20fff65e"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "12146bf5727e2056bf99a28fe94cde80"
server_key_iv: "3f253c12271de42e592fa672b23c74ee"
server_key_version: "encryptionkey@0"
I20260806 20:28:21.673486 32559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "65031948d14d47838c64d64d20fff65e"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "12146bf5727e2056bf99a28fe94cde80"
server_key_iv: "3f253c12271de42e592fa672b23c74ee"
server_key_version: "encryptionkey@0"
I20260806 20:28:21.679147 32559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 20:28:21.682897 32574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:21.684369 32559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:28:21.684548 32559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "65031948d14d47838c64d64d20fff65e"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "12146bf5727e2056bf99a28fe94cde80"
server_key_iv: "3f253c12271de42e592fa672b23c74ee"
server_key_version: "encryptionkey@0"
I20260806 20:28:21.684773 32559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:21.708297 32559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:21.713095 32559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:21.713464 32559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:21.714516 32559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:21.715975 32559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:21.716091 32559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:21.716259 32559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:21.716339 32559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:21.752349 32559 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:45529
I20260806 20:28:21.752413 32692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:45529 every 8 connection(s)
I20260806 20:28:21.753774 32559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:28:21.754518 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 32559
I20260806 20:28:21.754812 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:28:21.755693 17847 external_mini_cluster.cc:1428] Setting key 383e41df58540a7c95b388a5c366f4aa
Aug 06 20:28:21 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048101, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:21.778579 32696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:21.756853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:35053 (local address 127.17.109.254:34141)
0806 20:28:21.757391 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:21.757397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:21.758543 (+ 1146us) server_negotiation.cc:407] Connection header received
0806 20:28:21.761479 (+ 2936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:21.761483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:21.761551 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:21.761651 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:21.763763 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:21.764506 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:21.767069 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:21.767484 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:21.767671 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:21.772283 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:21.772288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:21.772292 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:21.772324 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:21.774982 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:21.776014 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:21.776022 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:21.776029 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:21.776203 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:21.776738 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:21.776742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:21.776744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:21.777022 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:21.777140 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:21.778118 (+ 978us) server_negotiation.cc:299] Negotiation successful
0806 20:28:21.778301 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":260,"threads_started":1}
I20260806 20:28:21.780184 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:21.757477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:35053)
0806 20:28:21.758340 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:21.758423 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:21.761162 (+ 2739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:21.761912 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:21.761925 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:21.762721 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:21.763547 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:21.763603 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:21.764720 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:21.764728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:21.766903 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:21.766921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:21.767767 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:21.767778 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:21.768119 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:21.768882 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:21.768918 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:21.772045 (+ 3127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:21.775130 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:21.775142 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:21.775162 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:21.775867 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:21.776331 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:21.776339 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:21.776345 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:21.776598 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:21.777164 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:21.777174 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:21.777679 (+ 505us) client_negotiation.cc:769] Sending connection context
0806 20:28:21.777995 (+ 316us) client_negotiation.cc:240] Negotiation successful
0806 20:28:21.778602 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":345,"threads_started":1}
I20260806 20:28:21.781919 32693 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34141
I20260806 20:28:21.782246 32693 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:21.782974 32693 heartbeater.cc:511] Master 127.17.109.254:34141 requested a full tablet report, sending...
I20260806 20:28:21.785418 32495 ts_manager.cc:194] Registered new tserver with Master: 65031948d14d47838c64d64d20fff65e (127.17.109.193:45529)
I20260806 20:28:21.787650 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:35053
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:28:21.825286 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34141
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:22.172619 32701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:22.173142 32701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:22.173331 32701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:22.184749 32701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:22.184859 32701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:22.184971 32701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:28:22.195137 32701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:22.196424 32701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:22.197728 32701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:22.211675 32706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:22.211773 32708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:22.211803 32710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:22.212466 32701 server_base.cc:1034] running on GCE node
I20260806 20:28:22.213099 32701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:22.213994 32701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:22.215193 32701 hybrid_clock.cc:648] HybridClock initialized: now 1786048102215151 us; error 38 us; skew 500 ppm
I20260806 20:28:22.215462 32701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:22 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:22.218623 32701 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:28:22.220114 32701 webserver.cc:460] Webserver started at http://127.17.109.194:43231/ using document root <none> and password file <none>
I20260806 20:28:22.220834 32701 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:22.220976 32701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:22.221382 32701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:22.224344 32701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "03df31deb75c41789903dc20b9c965f2"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "40ff8ff92eb2f4f47316f41a1169be13"
server_key_iv: "ff6aecdd7c185b33a39a248e74ba95d8"
server_key_version: "encryptionkey@0"
I20260806 20:28:22.225049 32701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "03df31deb75c41789903dc20b9c965f2"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "40ff8ff92eb2f4f47316f41a1169be13"
server_key_iv: "ff6aecdd7c185b33a39a248e74ba95d8"
server_key_version: "encryptionkey@0"
I20260806 20:28:22.230070 32701 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:28:22.233562 32718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:22.234838 32701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:28:22.235010 32701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "03df31deb75c41789903dc20b9c965f2"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "40ff8ff92eb2f4f47316f41a1169be13"
server_key_iv: "ff6aecdd7c185b33a39a248e74ba95d8"
server_key_version: "encryptionkey@0"
I20260806 20:28:22.235203 32701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:22.261788 32701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:22.266198 32701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:22.266453 32701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:22.267428 32701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:22.268834 32701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:22.268908 32701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:22.268992 32701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:22.269037 32701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:22.304055 32701 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:41967
I20260806 20:28:22.304104 368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:41967 every 8 connection(s)
I20260806 20:28:22.305363 32701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:28:22.315050 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 32701
I20260806 20:28:22.315330 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:28:22.316093 17847 external_mini_cluster.cc:1428] Setting key 6ad5a5d30498dede593cde303b439439
Aug 06 20:28:22 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:22.327426 371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.308207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:52729 (local address 127.17.109.254:34141)
0806 20:28:22.308675 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:22.308680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:22.309477 (+ 797us) server_negotiation.cc:407] Connection header received
0806 20:28:22.311338 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:22.311344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:22.311439 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:22.311561 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:22.313057 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.313777 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.315359 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.316124 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.316297 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.321087 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:22.321093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:22.321096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:22.321128 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:22.323905 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:22.324943 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:22.324947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:22.324949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:22.325081 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:22.325611 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:22.325614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:22.325616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:22.325887 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:22.325999 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:22.326986 (+ 987us) server_negotiation.cc:299] Negotiation successful
0806 20:28:22.327161 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":249,"threads_started":1}
I20260806 20:28:22.329401 372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.308599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:52729)
0806 20:28:22.309307 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:22.309347 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:22.311052 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:22.311816 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:22.311824 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:22.312336 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:22.312884 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.312901 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.313940 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.313944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.315149 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.315165 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.316392 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.316397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.316603 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.317484 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:22.317512 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:22.320774 (+ 3262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:22.324066 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:22.324079 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:22.324100 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:22.324819 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:22.325205 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:22.325214 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:22.325221 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:22.325491 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:22.326026 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:22.326038 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:22.326555 (+ 517us) client_negotiation.cc:769] Sending connection context
0806 20:28:22.326913 (+ 358us) client_negotiation.cc:240] Negotiation successful
0806 20:28:22.327585 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":267,"threads_started":1}
I20260806 20:28:22.330974 369 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34141
I20260806 20:28:22.331334 369 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:22.332327 369 heartbeater.cc:511] Master 127.17.109.254:34141 requested a full tablet report, sending...
I20260806 20:28:22.334081 32495 ts_manager.cc:194] Registered new tserver with Master: 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967)
I20260806 20:28:22.335093 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:52729
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:28:22.386652 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34141
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:22.725054 375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:22.725399 375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:22.725584 375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:22.732864 375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:22.733028 375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:22.733211 375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:28:22.743268 375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:22.744675 375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:22.746223 375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:22.761030 381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:22.761060 385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:22.761292 380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:22.761827 375 server_base.cc:1034] running on GCE node
I20260806 20:28:22.762406 375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:22.763442 375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:22.764698 375 hybrid_clock.cc:648] HybridClock initialized: now 1786048102764605 us; error 95 us; skew 500 ppm
I20260806 20:28:22.765019 375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:22 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:22.768298 375 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:28:22.769747 375 webserver.cc:460] Webserver started at http://127.17.109.195:44001/ using document root <none> and password file <none>
I20260806 20:28:22.770458 375 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:22.770592 375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:22.770911 375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:22.773965 375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "5dc8d2dfba966386f4017c07df73279b"
server_key_iv: "772204d61a3dbeb95d1b838be5bdac1c"
server_key_version: "encryptionkey@0"
I20260806 20:28:22.774766 375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "5dc8d2dfba966386f4017c07df73279b"
server_key_iv: "772204d61a3dbeb95d1b838be5bdac1c"
server_key_version: "encryptionkey@0"
I20260806 20:28:22.779983 375 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 20:28:22.783458 392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:22.784888 375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:28:22.785063 375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "5dc8d2dfba966386f4017c07df73279b"
server_key_iv: "772204d61a3dbeb95d1b838be5bdac1c"
server_key_version: "encryptionkey@0"
I20260806 20:28:22.785274 375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:22.791013 32693 heartbeater.cc:503] Master 127.17.109.254:34141 was elected leader, sending a full tablet report...
I20260806 20:28:22.808159 375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:22.813251 375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:22.813583 375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:22.814790 375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:22.816802 375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:22.816918 375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:22.817072 375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:22.817162 375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:22.865512 375 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:43441
I20260806 20:28:22.865624 509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:43441 every 8 connection(s)
I20260806 20:28:22.866859 375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:28:22.868724 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 375
I20260806 20:28:22.868923 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:28:22.869414 17847 external_mini_cluster.cc:1428] Setting key 77e2f8f590bc49acde2b562df5590db1
Aug 06 20:28:22 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:22.889497 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.870847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:41931 (local address 127.17.109.254:34141)
0806 20:28:22.871412 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:22.871418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:22.871974 (+ 556us) server_negotiation.cc:407] Connection header received
0806 20:28:22.873992 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:22.873997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:22.874099 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:22.874216 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:22.875742 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.876535 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.878063 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.878529 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.878672 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.882517 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:22.882526 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:22.882531 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:22.882579 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:22.886221 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:22.886972 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:22.886977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:22.886981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:22.887113 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:22.887542 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:22.887547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:22.887551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:22.887861 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:22.888029 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:22.888879 (+ 850us) server_negotiation.cc:299] Negotiation successful
0806 20:28:22.889096 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":276,"threads_started":1}
I20260806 20:28:22.890568 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.870336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:41931)
0806 20:28:22.871819 (+ 1483us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:22.871874 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:22.873742 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:22.874461 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:22.874470 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:22.874970 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:22.875526 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.875543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.876699 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.876704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.877912 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.877924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.878774 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.878781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.879024 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.879777 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:22.879805 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:22.882295 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:22.886375 (+ 4080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:22.886384 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:22.886401 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:22.886836 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:22.887227 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:22.887232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:22.887236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:22.887423 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:22.887991 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:22.887999 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:22.888405 (+ 406us) client_negotiation.cc:769] Sending connection context
0806 20:28:22.888768 (+ 363us) client_negotiation.cc:240] Negotiation successful
0806 20:28:22.889330 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"thread_start_us":898,"threads_started":1}
I20260806 20:28:22.892078 510 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34141
I20260806 20:28:22.892441 510 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:22.893383 510 heartbeater.cc:511] Master 127.17.109.254:34141 requested a full tablet report, sending...
I20260806 20:28:22.895031 32495 ts_manager.cc:194] Registered new tserver with Master: ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441)
I20260806 20:28:22.896324 32495 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:41931
I20260806 20:28:22.898036 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:28:22.916069 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.903033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.17.109.254:34141)
0806 20:28:22.903238 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:22.903247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:22.903794 (+ 547us) server_negotiation.cc:407] Connection header received
0806 20:28:22.903993 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:22.903999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:22.904084 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:22.904196 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:22.905552 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.906289 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.907638 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.908040 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.908190 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.910650 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:22.910659 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:22.910663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:22.910701 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:22.913362 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:22.914045 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:22.914052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:22.914055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:22.914194 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:22.914595 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:22.914600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:22.914603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:22.914920 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:22.915186 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:22.915507 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 20:28:22.915755 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 20:28:22.920401 32495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:28:22.927217 32495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:28:22.962656 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.940108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:41967 (local address 127.0.0.1:59790)
0806 20:28:22.952234 (+ 12126us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:22.952276 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:22.952477 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:22.953216 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:22.953225 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:22.953258 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:22.953817 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.953834 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.956099 (+ 2265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.956105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.959080 (+ 2975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.959096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.961563 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.961569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.961820 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.961885 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 20:28:22.962043 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 20:28:22.962243 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11848,"thread_start_us":13505,"threads_started":1}
I20260806 20:28:22.963897 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.940377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.17.109.194:41967)
0806 20:28:22.951314 (+ 10937us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:22.951329 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:22.952667 (+ 1338us) server_negotiation.cc:407] Connection header received
0806 20:28:22.952760 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:22.952766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:22.952980 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:22.953216 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:22.954076 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.955914 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.959292 (+ 3378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.960975 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.961149 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.961354 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:22.963290 (+ 1936us) server_negotiation.cc:299] Negotiation successful
0806 20:28:22.963489 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10680,"thread_start_us":10968,"threads_started":1}
I20260806 20:28:22.964277 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.940355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:43441 (local address 127.0.0.1:38088)
0806 20:28:22.953946 (+ 13591us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:22.953966 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:22.954076 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:22.955597 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:22.955603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:22.955629 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:22.956077 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.956090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.958465 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.958470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.961000 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.961016 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.963377 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.963384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.963604 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.963649 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 20:28:22.963736 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 20:28:22.963874 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13436,"thread_start_us":11442,"threads_started":1}
I20260806 20:28:22.965217 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.941250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.17.109.195:43441)
0806 20:28:22.954703 (+ 13453us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:22.954712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:22.954753 (+ 41us) server_negotiation.cc:407] Connection header received
0806 20:28:22.954846 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:22.954854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:22.955076 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:22.955290 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:22.956370 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.958234 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.961218 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.962796 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.963000 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.963212 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:22.964803 (+ 1591us) server_negotiation.cc:299] Negotiation successful
0806 20:28:22.964942 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13256,"thread_start_us":13085,"threads_started":1}
I20260806 20:28:22.969503 443 tablet_service.cc:1467] Processing CreateTablet for tablet 2ef2df14dcae46ee935a6d3a4cb727d6 (DEFAULT_TABLE table=test-table [id=2606eeb439f5488fb33c03213dd89cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:22.971276 443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ef2df14dcae46ee935a6d3a4cb727d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:22.970903 301 tablet_service.cc:1467] Processing CreateTablet for tablet 2ef2df14dcae46ee935a6d3a4cb727d6 (DEFAULT_TABLE table=test-table [id=2606eeb439f5488fb33c03213dd89cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:22.972327 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ef2df14dcae46ee935a6d3a4cb727d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:22.975108 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.954700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60190)
0806 20:28:22.962972 (+ 8272us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:22.962992 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:22.963104 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:22.967852 (+ 4748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:22.967857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:22.967886 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:22.968068 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.968079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.970112 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.970118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.972286 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:22.972306 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.974262 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.974269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:22.974491 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.974531 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 20:28:22.974630 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 20:28:22.974790 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8164,"thread_start_us":12341,"threads_started":1}
I20260806 20:28:22.975912 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:22.954416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60190 (local address 127.17.109.193:45529)
0806 20:28:22.966864 (+ 12448us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:22.966874 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:22.966905 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:28:22.967021 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:22.967029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:22.967304 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:22.967583 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:22.968281 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.969928 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.972532 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:22.974042 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:22.974262 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:22.974426 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:22.975321 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 20:28:22.975522 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12258,"thread_start_us":12110,"threads_started":1}
I20260806 20:28:22.980084 32628 tablet_service.cc:1467] Processing CreateTablet for tablet 2ef2df14dcae46ee935a6d3a4cb727d6 (DEFAULT_TABLE table=test-table [id=2606eeb439f5488fb33c03213dd89cc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:22.981685 32628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ef2df14dcae46ee935a6d3a4cb727d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:22.986442 529 tablet_bootstrap.cc:492] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Bootstrap starting.
I20260806 20:28:22.988397 530 tablet_bootstrap.cc:492] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Bootstrap starting.
I20260806 20:28:22.990653 529 tablet_bootstrap.cc:654] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:22.992477 529 log.cc:824] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:22.992551 530 tablet_bootstrap.cc:654] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:22.993799 530 log.cc:824] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:22.996001 529 tablet_bootstrap.cc:492] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: No bootstrap required, opened a new log
I20260806 20:28:22.996408 529 ts_tablet_manager.cc:1397] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260806 20:28:22.996455 534 tablet_bootstrap.cc:492] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: Bootstrap starting.
I20260806 20:28:22.997169 530 tablet_bootstrap.cc:492] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: No bootstrap required, opened a new log
I20260806 20:28:22.997473 530 ts_tablet_manager.cc:1397] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 20:28:23.000000 534 tablet_bootstrap.cc:654] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:23.001243 534 log.cc:824] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:23.004033 534 tablet_bootstrap.cc:492] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: No bootstrap required, opened a new log
I20260806 20:28:23.004316 534 ts_tablet_manager.cc:1397] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 20:28:23.009544 529 raft_consensus.cc:348] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.009544 530 raft_consensus.cc:348] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.010063 529 raft_consensus.cc:374] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:23.010063 530 raft_consensus.cc:374] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:23.010190 529 raft_consensus.cc:729] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8f3f0a55fd4d9695c33c1e7f19196f, State: Initialized, Role: FOLLOWER
I20260806 20:28:23.010179 530 raft_consensus.cc:729] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03df31deb75c41789903dc20b9c965f2, State: Initialized, Role: FOLLOWER
I20260806 20:28:23.010890 529 consensus_queue.cc:260] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.010890 530 consensus_queue.cc:260] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.012315 369 heartbeater.cc:503] Master 127.17.109.254:34141 was elected leader, sending a full tablet report...
I20260806 20:28:23.013505 529 ts_tablet_manager.cc:1428] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260806 20:28:23.013892 510 heartbeater.cc:503] Master 127.17.109.254:34141 was elected leader, sending a full tablet report...
I20260806 20:28:23.014990 530 ts_tablet_manager.cc:1428] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Time spent starting tablet: real 0.017s user 0.009s sys 0.008s
I20260806 20:28:23.015806 534 raft_consensus.cc:348] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.016283 534 raft_consensus.cc:374] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:23.016409 534 raft_consensus.cc:729] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65031948d14d47838c64d64d20fff65e, State: Initialized, Role: FOLLOWER
I20260806 20:28:23.017045 534 consensus_queue.cc:260] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.019791 534 ts_tablet_manager.cc:1428] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
W20260806 20:28:23.057543 370 tablet.cc:2369] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:28:23.074265 537 raft_consensus.cc:482] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:23.074628 537 raft_consensus.cc:504] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.076565 537 leader_election.cc:288] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967), 65031948d14d47838c64d64d20fff65e (127.17.109.193:45529)
I20260806 20:28:23.083364 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:23.076791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:41967 (local address 127.17.109.195:34735)
0806 20:28:23.077015 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:23.077038 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:23.077203 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:23.077776 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:23.077785 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:23.077824 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:23.078069 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:23.078086 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.079339 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.079343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:23.081490 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:23.081506 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.082773 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.082777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:23.082918 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:23.082944 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 20:28:23.083024 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 20:28:23.083123 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":1}
I20260806 20:28:23.083469 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:23.077060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:34735 (local address 127.17.109.194:41967)
0806 20:28:23.077379 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:23.077388 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:23.077415 (+ 27us) server_negotiation.cc:407] Connection header received
0806 20:28:23.077512 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:23.077521 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:23.077623 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:23.077724 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:23.078234 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.079190 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.081655 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.082606 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.082774 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:23.082849 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:23.083080 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 20:28:23.083213 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260806 20:28:23.084506 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:23.077311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:52753 (local address 127.17.109.193:45529)
0806 20:28:23.077575 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:23.077583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:23.078108 (+ 525us) server_negotiation.cc:407] Connection header received
0806 20:28:23.078310 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:23.078315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:23.078408 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:23.078507 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:23.079038 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.079896 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.082231 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.083443 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.083638 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:23.083714 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:23.084126 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 20:28:23.084267 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 20:28:23.084587 539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:23.077188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.17.109.195:52753)
0806 20:28:23.077992 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:23.078014 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:23.078148 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:23.078527 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:23.078531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:23.078551 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:23.078888 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:23.078897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.080057 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.080063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:23.082077 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:23.082089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.083848 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.083852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:23.083968 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:23.083990 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 20:28:23.084098 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 20:28:23.084221 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":1,"thread_start_us":379,"threads_started":1}
I20260806 20:28:23.084518 322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ef2df14dcae46ee935a6d3a4cb727d6" candidate_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03df31deb75c41789903dc20b9c965f2" is_pre_election: true
I20260806 20:28:23.085256 322 raft_consensus.cc:2454] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca8f3f0a55fd4d9695c33c1e7f19196f in term 0.
I20260806 20:28:23.085645 32648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ef2df14dcae46ee935a6d3a4cb727d6" candidate_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65031948d14d47838c64d64d20fff65e" is_pre_election: true
I20260806 20:28:23.086051 393 leader_election.cc:302] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03df31deb75c41789903dc20b9c965f2, ca8f3f0a55fd4d9695c33c1e7f19196f; no voters:
I20260806 20:28:23.086133 32648 raft_consensus.cc:2454] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca8f3f0a55fd4d9695c33c1e7f19196f in term 0.
I20260806 20:28:23.086524 537 raft_consensus.cc:2781] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:28:23.086745 537 raft_consensus.cc:482] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:28:23.086879 537 raft_consensus.cc:3037] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:23.089941 537 raft_consensus.cc:504] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.090611 537 leader_election.cc:288] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [CANDIDATE]: Term 1 election: Requested vote from peers 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967), 65031948d14d47838c64d64d20fff65e (127.17.109.193:45529)
I20260806 20:28:23.091094 322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ef2df14dcae46ee935a6d3a4cb727d6" candidate_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03df31deb75c41789903dc20b9c965f2"
I20260806 20:28:23.091216 32648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ef2df14dcae46ee935a6d3a4cb727d6" candidate_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65031948d14d47838c64d64d20fff65e"
I20260806 20:28:23.091324 322 raft_consensus.cc:3037] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:23.091483 32648 raft_consensus.cc:3037] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:23.094108 32648 raft_consensus.cc:2454] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8f3f0a55fd4d9695c33c1e7f19196f in term 1.
I20260806 20:28:23.094204 322 raft_consensus.cc:2454] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca8f3f0a55fd4d9695c33c1e7f19196f in term 1.
I20260806 20:28:23.094534 393 leader_election.cc:302] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65031948d14d47838c64d64d20fff65e, ca8f3f0a55fd4d9695c33c1e7f19196f; no voters:
I20260806 20:28:23.094859 537 raft_consensus.cc:2781] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:23.095329 537 raft_consensus.cc:686] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 LEADER]: Becoming Leader. State: Replica: ca8f3f0a55fd4d9695c33c1e7f19196f, State: Running, Role: LEADER
I20260806 20:28:23.095928 537 consensus_queue.cc:237] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } }
I20260806 20:28:23.100746 32495 catalog_manager.cc:5495] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f reported cstate change: term changed from 0 to 1, leader changed from <none> to ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195). New cstate: current_term: 1 leader_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } health_report { overall_health: UNKNOWN } } }
W20260806 20:28:23.118541 511 tablet.cc:2369] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:28:23.136514 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:23.130934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38098 (local address 127.17.109.195:43441)
0806 20:28:23.131129 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:23.131134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:23.131183 (+ 49us) server_negotiation.cc:407] Connection header received
0806 20:28:23.131389 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:23.131393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:23.131488 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:23.131713 (+ 225us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:28:23.132085 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.132960 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.134379 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.134818 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.134917 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:23.135362 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:28:23.136014 (+ 652us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:28:23.136134 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:23.136220 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 20:28:23.136291 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 20:28:23.145552 322 raft_consensus.cc:1261] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 1 FOLLOWER]: Refusing update from remote peer ca8f3f0a55fd4d9695c33c1e7f19196f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:23.145519 32648 raft_consensus.cc:1261] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Refusing update from remote peer ca8f3f0a55fd4d9695c33c1e7f19196f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:23.146605 540 consensus_queue.cc:1035] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [LEADER]: Connected to new peer: Peer: permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:23.147013 537 consensus_queue.cc:1035] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [LEADER]: Connected to new peer: Peer: permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:23.160429 543 mvcc.cc:205] Tried to move back new op lower bound from 7315653030472511488 to 7315653030285180928. Current Snapshot: MvccSnapshot[applied={T|T < 7315653030472511488}]
I20260806 20:28:23.177587 32494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:28:23.177970 32494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:28:23.182405 32494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Sending DeleteTablet for 3 replicas of tablet 2ef2df14dcae46ee935a6d3a4cb727d6
I20260806 20:28:23.183686 301 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ef2df14dcae46ee935a6d3a4cb727d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:23 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:59790
I20260806 20:28:23.183746 443 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ef2df14dcae46ee935a6d3a4cb727d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:23 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:38088
I20260806 20:28:23.184128 32628 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ef2df14dcae46ee935a6d3a4cb727d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:23 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:60190
I20260806 20:28:23.185976 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 32559
I20260806 20:28:23.193770 555 tablet_replica.cc:330] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: stopping tablet replica
I20260806 20:28:23.194303 555 raft_consensus.cc:2229] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:28:23.194940 555 raft_consensus.cc:2258] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:23.199666 555 ts_tablet_manager.cc:1905] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:23.208976 554 tablet_replica.cc:330] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: stopping tablet replica
I20260806 20:28:23.209514 554 raft_consensus.cc:2229] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:28:23.210374 554 raft_consensus.cc:2258] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:23.212848 553 tablet_replica.cc:330] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: stopping tablet replica
I20260806 20:28:23.213264 555 ts_tablet_manager.cc:1918] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:23.213460 555 log.cc:1196] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/2ef2df14dcae46ee935a6d3a4cb727d6
I20260806 20:28:23.213496 553 raft_consensus.cc:2229] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:28:23.214105 555 ts_tablet_manager.cc:1939] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 65031948d14d47838c64d64d20fff65e: Deleting consensus metadata
I20260806 20:28:23.214350 553 raft_consensus.cc:2258] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:23.216287 32479 catalog_manager.cc:4841] TS 65031948d14d47838c64d64d20fff65e (127.17.109.193:45529): tablet 2ef2df14dcae46ee935a6d3a4cb727d6 (table test-table [id=2606eeb439f5488fb33c03213dd89cc8]) successfully deleted
Aug 06 20:28:23 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:28:23.216995 554 ts_tablet_manager.cc:1905] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:23.219341 553 ts_tablet_manager.cc:1905] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:23.224529 554 ts_tablet_manager.cc:1918] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:23.224722 554 log.cc:1196] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/2ef2df14dcae46ee935a6d3a4cb727d6
I20260806 20:28:23.225327 554 ts_tablet_manager.cc:1939] T 2ef2df14dcae46ee935a6d3a4cb727d6 P ca8f3f0a55fd4d9695c33c1e7f19196f: Deleting consensus metadata
I20260806 20:28:23.227330 32482 catalog_manager.cc:4841] TS ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441): tablet 2ef2df14dcae46ee935a6d3a4cb727d6 (table test-table [id=2606eeb439f5488fb33c03213dd89cc8]) successfully deleted
I20260806 20:28:23.227540 553 ts_tablet_manager.cc:1918] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:23.227746 553 log.cc:1196] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/2ef2df14dcae46ee935a6d3a4cb727d6
I20260806 20:28:23.227780 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:23.187637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60204 (local address 127.17.109.193:45529)
0806 20:28:23.187889 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:23.187899 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:23.200694 (+ 12795us) server_negotiation.cc:407] Connection header received
0806 20:28:23.201072 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:23.201082 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:23.201183 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:23.201295 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:23.204401 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.205956 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.209468 (+ 3512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.210391 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.210520 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:23.218166 (+ 7646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:23.218193 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:23.218211 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:23.218260 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:23.222775 (+ 4515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:23.223768 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:23.223777 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:23.223781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:23.223950 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:23.224500 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:23.224507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:23.224510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:23.225159 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:23.226731 (+ 1572us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:23.226923 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 20:28:23.227189 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 20:28:23.228348 553 ts_tablet_manager.cc:1939] T 2ef2df14dcae46ee935a6d3a4cb727d6 P 03df31deb75c41789903dc20b9c965f2: Deleting consensus metadata
I20260806 20:28:23.228720 32688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:23.230360 32479 catalog_manager.cc:4841] TS 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967): tablet 2ef2df14dcae46ee935a6d3a4cb727d6 (table test-table [id=2606eeb439f5488fb33c03213dd89cc8]) successfully deleted
I20260806 20:28:23.328577 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 32559
I20260806 20:28:23.351177 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 32701
Aug 06 20:28:23 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:28:23.369174 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:23.352019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.17.109.194:41967)
0806 20:28:23.352225 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:23.352230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:23.352420 (+ 190us) server_negotiation.cc:407] Connection header received
0806 20:28:23.352613 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:23.352617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:23.352692 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:23.352798 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:23.354611 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.355641 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.357012 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.357743 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.357889 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:23.362800 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:23.362817 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:23.362832 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:23.362869 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:23.365841 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:23.366756 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:23.366763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:23.366765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:23.366891 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:23.367332 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:23.367335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:23.367337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:23.368044 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:23.368273 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:23.368697 (+ 424us) server_negotiation.cc:299] Negotiation successful
0806 20:28:23.368872 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 20:28:23.370095 364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:23.462705 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 32701
I20260806 20:28:23.483227 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 375
Aug 06 20:28:23 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:28:23.500829 526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:23.484120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.17.109.195:43441)
0806 20:28:23.484336 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:23.484342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:23.484452 (+ 110us) server_negotiation.cc:407] Connection header received
0806 20:28:23.484661 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:23.484673 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:23.484778 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:23.484891 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:23.486244 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.487039 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.488614 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:23.489134 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:23.489322 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:23.493997 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:23.494010 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:23.494024 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:23.494065 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:23.497750 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:23.498552 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:23.498560 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:23.498562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:23.498713 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:23.499107 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:23.499112 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:23.499115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:23.499513 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:23.499871 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:23.500184 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 20:28:23.500434 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260806 20:28:23.501699 505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:23.625123 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 375
I20260806 20:28:23.646051 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 32463
I20260806 20:28:23.647344 32526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:23.758759 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 32463
I20260806 20:28:23.784930 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34141
--webserver_interface=127.17.109.254
--webserver_port=43353
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:24.136126 563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:24.136443 563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:24.136636 563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:24.144084 563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:24.144193 563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:24.144322 563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:24.144447 563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:24.144529 563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:24.154527 563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=43353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:24.155933 563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:24.157322 563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:24.187209 570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:24.187242 571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:24.187400 573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:24.187902 563 server_base.cc:1034] running on GCE node
I20260806 20:28:24.189018 563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:24.190716 563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:24.191993 563 hybrid_clock.cc:648] HybridClock initialized: now 1786048104191931 us; error 44 us; skew 500 ppm
I20260806 20:28:24.192425 563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:24 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048104, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:24.196166 563 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:28:24.197959 563 webserver.cc:460] Webserver started at http://127.17.109.254:43353/ using document root <none> and password file <none>
I20260806 20:28:24.198761 563 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:24.198935 563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:24.205451 563 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 20:28:24.208746 581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:24.210124 563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:28:24.210330 563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "d4f8112ed6524564b85abd04865e4f3f"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "7d4ddf9d8bed820339fd56f71017d557"
server_key_iv: "a1bc55c373dd1adc2921a2d69caf2257"
server_key_version: "encryptionkey@0"
I20260806 20:28:24.211397 563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:24.232932 563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:24.237298 563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:24.237604 563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:24.258291 563 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:34141
I20260806 20:28:24.258340 637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:34141 every 8 connection(s)
I20260806 20:28:24.259950 563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:28:24.263114 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 563
I20260806 20:28:24.264034 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:45529
--local_ip_for_outbound_sockets=127.17.109.193
--tserver_master_addrs=127.17.109.254:34141
--webserver_port=41751
--webserver_interface=127.17.109.193
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:28:24.268187 638 sys_catalog.cc:263] Verifying existing consensus state
I20260806 20:28:24.271988 638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Bootstrap starting.
I20260806 20:28:24.302722 638 log.cc:824] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:24.320397 638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:28:24.321033 638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Bootstrap complete.
I20260806 20:28:24.330529 638 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:24.331511 638 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f8112ed6524564b85abd04865e4f3f, State: Initialized, Role: FOLLOWER
I20260806 20:28:24.332192 638 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:24.332451 638 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:28:24.332592 638 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:28:24.332794 638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 1 FOLLOWER]: Advancing to term 2
I20260806 20:28:24.336823 638 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:24.337424 638 leader_election.cc:302] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4f8112ed6524564b85abd04865e4f3f; no voters:
I20260806 20:28:24.338076 638 leader_election.cc:288] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 20:28:24.338303 642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 2 FOLLOWER]: Leader election won for term 2
I20260806 20:28:24.339578 642 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 2 LEADER]: Becoming Leader. State: Replica: d4f8112ed6524564b85abd04865e4f3f, State: Running, Role: LEADER
I20260806 20:28:24.340167 642 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:24.340444 638 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:24.342598 644 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4f8112ed6524564b85abd04865e4f3f. Latest consensus state: current_term: 2 leader_uuid: "d4f8112ed6524564b85abd04865e4f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } } }
I20260806 20:28:24.342988 644 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:24.343593 643 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d4f8112ed6524564b85abd04865e4f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } } }
I20260806 20:28:24.343902 643 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:24.352229 647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:28:24.360564 647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:28:24.362046 647 catalog_manager.cc:1259] Loaded cluster ID: 6bea52ace37d4f508c40c163459470b3
I20260806 20:28:24.362154 647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:28:24.365391 647 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 20:28:24.384503 647 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 20:28:24.403764 647 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:28:24.405154 647 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Loaded TSK: 0
I20260806 20:28:24.406358 647 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 20:28:24.662922 640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:24.663203 640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:24.663349 640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:24.671402 640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:24.671518 640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:24.671742 640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:28:24.681915 640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:45529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=41751
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:24.683283 640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:24.684679 640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:24.700450 670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:24.701378 667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:24.700433 668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:24.702589 640 server_base.cc:1034] running on GCE node
I20260806 20:28:24.703236 640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:24.704334 640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:24.705557 640 hybrid_clock.cc:648] HybridClock initialized: now 1786048104705512 us; error 38 us; skew 500 ppm
I20260806 20:28:24.705955 640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:24 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048104, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:24.716085 640 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:28:24.717702 640 webserver.cc:460] Webserver started at http://127.17.109.193:41751/ using document root <none> and password file <none>
I20260806 20:28:24.718529 640 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:24.718703 640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:24.724385 640 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 20:28:24.727422 677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:24.729061 640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:28:24.729250 640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "65031948d14d47838c64d64d20fff65e"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "12146bf5727e2056bf99a28fe94cde80"
server_key_iv: "3f253c12271de42e592fa672b23c74ee"
server_key_version: "encryptionkey@0"
I20260806 20:28:24.730237 640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:24.750994 640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:24.756254 640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:24.756541 640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:24.757608 640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:24.759079 640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:24.759193 640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:24.759320 640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:24.759419 640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:24.791458 640 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:45529
I20260806 20:28:24.791569 801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:45529 every 8 connection(s)
I20260806 20:28:24.792969 640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:28:24.800213 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 640
I20260806 20:28:24.801044 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:41967
--local_ip_for_outbound_sockets=127.17.109.194
--tserver_master_addrs=127.17.109.254:34141
--webserver_port=43231
--webserver_interface=127.17.109.194
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:28:24 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048104, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:24.820000 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:24.796029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:50295 (local address 127.17.109.254:34141)
0806 20:28:24.796765 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:24.796776 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:24.797087 (+ 311us) server_negotiation.cc:407] Connection header received
0806 20:28:24.799190 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:24.799204 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:24.799697 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:24.800010 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:24.802108 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:24.803380 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:24.805879 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:24.806871 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:24.807056 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:24.811726 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:24.811739 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:24.811761 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:24.811814 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:24.814914 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:24.815981 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:24.815991 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:24.816008 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:24.816293 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:24.816800 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:24.816804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:24.816806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:24.817351 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:24.817519 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:24.818510 (+ 991us) server_negotiation.cc:299] Negotiation successful
0806 20:28:24.818868 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":339,"threads_started":1}
I20260806 20:28:24.820289 805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:24.796071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:50295)
0806 20:28:24.796906 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:24.796954 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:24.798664 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:24.800241 (+ 1577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:24.800254 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:24.801058 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:24.801869 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:24.801897 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:24.803578 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:24.803586 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:24.805703 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:24.805721 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:24.807120 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:24.807129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:24.807469 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:24.808318 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:24.808352 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:24.811401 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:24.815062 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:24.815074 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:24.815093 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:24.815788 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:24.816415 (+ 627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:24.816423 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:24.816429 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:24.816670 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:24.817506 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:24.817517 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:24.817962 (+ 445us) client_negotiation.cc:769] Sending connection context
0806 20:28:24.818277 (+ 315us) client_negotiation.cc:240] Negotiation successful
0806 20:28:24.818888 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":340,"threads_started":1}
I20260806 20:28:24.824234 802 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34141
I20260806 20:28:24.824604 802 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:24.825453 802 heartbeater.cc:511] Master 127.17.109.254:34141 requested a full tablet report, sending...
I20260806 20:28:24.827929 600 ts_manager.cc:194] Registered new tserver with Master: 65031948d14d47838c64d64d20fff65e (127.17.109.193:45529)
I20260806 20:28:24.829416 600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:50295
W20260806 20:28:25.164631 806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:25.165051 806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:25.165217 806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:25.174177 806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:25.174335 806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:25.174535 806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:28:25.184597 806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:41967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=43231
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:25.185927 806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:25.187332 806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:25.200572 811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:25.200646 812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:25.200870 814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:25.201831 806 server_base.cc:1034] running on GCE node
I20260806 20:28:25.202454 806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:25.203379 806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:25.204667 806 hybrid_clock.cc:648] HybridClock initialized: now 1786048105204661 us; error 123 us; skew 500 ppm
I20260806 20:28:25.205094 806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:25 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048105, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:25.208333 806 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:28:25.209792 806 webserver.cc:460] Webserver started at http://127.17.109.194:43231/ using document root <none> and password file <none>
I20260806 20:28:25.210443 806 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:25.210558 806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:25.215969 806 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:28:25.218847 821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:25.220680 806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:28:25.220844 806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "03df31deb75c41789903dc20b9c965f2"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "40ff8ff92eb2f4f47316f41a1169be13"
server_key_iv: "ff6aecdd7c185b33a39a248e74ba95d8"
server_key_version: "encryptionkey@0"
I20260806 20:28:25.221884 806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:25.246865 806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:25.251058 806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:25.251317 806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:25.252429 806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:25.254035 806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:25.254148 806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:25.254235 806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:25.254299 806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:25.292748 806 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:41967
I20260806 20:28:25.292851 942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:41967 every 8 connection(s)
I20260806 20:28:25.294248 806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:28:25.294838 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 806
I20260806 20:28:25.295717 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:43441
--local_ip_for_outbound_sockets=127.17.109.195
--tserver_master_addrs=127.17.109.254:34141
--webserver_port=44001
--webserver_interface=127.17.109.195
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:28:25 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048105, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:25.315976 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.297108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:34331 (local address 127.17.109.254:34141)
0806 20:28:25.297343 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:25.297352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:25.298548 (+ 1196us) server_negotiation.cc:407] Connection header received
0806 20:28:25.300905 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:25.300915 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:25.301036 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:25.301162 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:25.302620 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.303774 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.305271 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.306028 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.306170 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.310130 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:25.310135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:25.310138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:25.310183 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:25.312822 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:25.313573 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:25.313582 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:25.313587 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:25.313734 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:25.314172 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:25.314181 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:25.314183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:25.314444 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:25.314603 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:25.315338 (+ 735us) server_negotiation.cc:299] Negotiation successful
0806 20:28:25.315610 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 20:28:25.316937 945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.297533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:34331)
0806 20:28:25.298334 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:25.298414 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:25.300610 (+ 2196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:25.301391 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:25.301399 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:25.301879 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:25.302445 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.302462 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.303932 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.303936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.305120 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.305131 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.306267 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.306276 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.306582 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.307360 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:25.307387 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:25.309906 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:25.312980 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:25.312988 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:25.313003 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:25.313441 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:25.313848 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:25.313852 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:25.313856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:25.314046 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:25.314576 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:25.314583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:25.314947 (+ 364us) client_negotiation.cc:769] Sending connection context
0806 20:28:25.315219 (+ 272us) client_negotiation.cc:240] Negotiation successful
0806 20:28:25.315825 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":341,"threads_started":1}
I20260806 20:28:25.318444 943 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34141
I20260806 20:28:25.318769 943 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:25.319526 943 heartbeater.cc:511] Master 127.17.109.254:34141 requested a full tablet report, sending...
I20260806 20:28:25.321178 600 ts_manager.cc:194] Registered new tserver with Master: 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967)
I20260806 20:28:25.321981 600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:34331
W20260806 20:28:25.640615 946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:25.641268 946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:25.641420 946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:25.649040 946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:25.649168 946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:25.649300 946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:28:25.659485 946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:43441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=44001
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:25.660800 946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:25.662230 946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:25.676219 952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:25.676124 951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:25.676077 954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:25.677278 946 server_base.cc:1034] running on GCE node
I20260806 20:28:25.677903 946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:25.678994 946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:25.680245 946 hybrid_clock.cc:648] HybridClock initialized: now 1786048105680176 us; error 51 us; skew 500 ppm
I20260806 20:28:25.680594 946 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:25 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048105, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:25.684049 946 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:28:25.685662 946 webserver.cc:460] Webserver started at http://127.17.109.195:44001/ using document root <none> and password file <none>
I20260806 20:28:25.686378 946 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:25.686556 946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:25.692180 946 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 20:28:25.694995 961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:25.696504 946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:28:25.696707 946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "5dc8d2dfba966386f4017c07df73279b"
server_key_iv: "772204d61a3dbeb95d1b838be5bdac1c"
server_key_version: "encryptionkey@0"
I20260806 20:28:25.697786 946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:25.719797 946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:25.724840 946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:25.725191 946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:25.726352 946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:25.727922 946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:25.728063 946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:25.728227 946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:25.728327 946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:25.765383 946 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:43441
I20260806 20:28:25.765437 1081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:43441 every 8 connection(s)
I20260806 20:28:25.767014 946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:28:25.767413 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 946
Aug 06 20:28:25 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048105, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:25.786288 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.768534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.17.109.254:34141)
0806 20:28:25.768770 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:25.768777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:25.769519 (+ 742us) server_negotiation.cc:407] Connection header received
0806 20:28:25.769751 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:25.769756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:25.769856 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:25.769970 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:25.772310 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.773463 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.775000 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.775659 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.775791 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.779160 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:25.779165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:25.779168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:25.779201 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:25.782856 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:25.783840 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:25.783844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:25.783847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:25.783984 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:25.784471 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:25.784477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:25.784481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:25.784941 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:25.785250 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:25.785652 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 20:28:25.785925 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 20:28:25.791334 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.769965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:50541 (local address 127.17.109.254:34141)
0806 20:28:25.770424 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:25.770428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:25.771676 (+ 1248us) server_negotiation.cc:407] Connection header received
0806 20:28:25.774019 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:25.774024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:25.774109 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:25.774228 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:25.776011 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.777316 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.778911 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.779519 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.779815 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.784278 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:25.784283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:25.784286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:25.784315 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:25.787744 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:25.788718 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:25.788722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:25.788724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:25.788912 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:25.789458 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:25.789462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:25.789464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:25.789789 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:25.789908 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:25.790598 (+ 690us) server_negotiation.cc:299] Negotiation successful
0806 20:28:25.790882 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":265,"threads_started":1}
I20260806 20:28:25.792055 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.770697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:50541)
0806 20:28:25.771488 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:25.771531 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:25.773778 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:25.774465 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:25.774473 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:25.775000 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:25.775808 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.775834 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.777484 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.777488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.778717 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.778729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.779949 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.779958 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.780287 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.780969 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:25.781010 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:25.784011 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:25.787894 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:25.787903 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:25.787925 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:25.788555 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:25.789054 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:25.789059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:25.789062 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:25.789341 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:25.789908 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:25.789914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:25.790237 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 20:28:25.790457 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 20:28:25.790842 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":372,"threads_started":1}
I20260806 20:28:25.793532 1082 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34141
I20260806 20:28:25.793879 1082 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:25.794824 1082 heartbeater.cc:511] Master 127.17.109.254:34141 requested a full tablet report, sending...
I20260806 20:28:25.796561 600 ts_manager.cc:194] Registered new tserver with Master: ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441)
I20260806 20:28:25.797434 600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:50541
I20260806 20:28:25.801764 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:28:25.808918 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.802815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.17.109.254:34141)
0806 20:28:25.802985 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:25.802989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:25.803469 (+ 480us) server_negotiation.cc:407] Connection header received
0806 20:28:25.803806 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:25.803810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:25.803939 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:25.804094 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:28:25.804560 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.805249 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.806817 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.807202 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.807343 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.807886 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:28:25.808336 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:28:25.808506 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:25.808564 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 20:28:25.808641 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 20:28:25.809746 600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:28:25.817123 600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:28:25.832556 802 heartbeater.cc:503] Master 127.17.109.254:34141 was elected leader, sending a full tablet report...
I20260806 20:28:25.844336 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.828989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60210)
0806 20:28:25.831386 (+ 2397us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:25.831439 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:25.831730 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:25.832999 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:25.834234 (+ 1235us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:25.834275 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:25.834920 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.834943 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.838342 (+ 3399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.838352 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.841002 (+ 2650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.841024 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.843238 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.843245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.843445 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.843498 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 20:28:25.843662 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 20:28:25.843899 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2064,"thread_start_us":532,"threads_started":1}
I20260806 20:28:25.844645 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.829285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.17.109.193:45529)
0806 20:28:25.830426 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:25.830433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:25.831987 (+ 1554us) server_negotiation.cc:407] Connection header received
0806 20:28:25.832126 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:25.832139 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:25.832442 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:25.832648 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:25.835231 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.838110 (+ 2879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.841228 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.843017 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.843223 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.843402 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:25.844070 (+ 668us) server_negotiation.cc:299] Negotiation successful
0806 20:28:25.844279 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":804,"threads_started":1}
I20260806 20:28:25.848855 730 tablet_service.cc:1467] Processing CreateTablet for tablet 999d1560782342748934c0df1c624311 (DEFAULT_TABLE table=test-table [id=e2639d882a1d4b0790cc56d2c7a58e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:25.850159 730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999d1560782342748934c0df1c624311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:25.851277 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.829447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:43441 (local address 127.0.0.1:38124)
0806 20:28:25.839978 (+ 10531us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:25.840005 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:25.840165 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:25.843996 (+ 3831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:25.844004 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:25.844042 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:25.844461 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.844479 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.846248 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.846254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.848397 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.848413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.850472 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.850481 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.850661 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.850697 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 20:28:25.850799 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 20:28:25.850926 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10359,"thread_start_us":10174,"threads_started":1}
I20260806 20:28:25.851444 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.830046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38124 (local address 127.17.109.195:43441)
0806 20:28:25.843063 (+ 13017us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:25.843070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:25.843126 (+ 56us) server_negotiation.cc:407] Connection header received
0806 20:28:25.843243 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:25.843250 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:25.843497 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:25.843716 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:25.844683 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.846068 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.848573 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.850138 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.850358 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.850540 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:25.850877 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 20:28:25.851071 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12858,"thread_start_us":12705,"threads_started":1}
I20260806 20:28:25.852396 1096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.831207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:41967 (local address 127.0.0.1:59812)
0806 20:28:25.842664 (+ 11457us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:25.842686 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:25.842818 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:25.843656 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:25.843661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:25.843700 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:25.844217 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.844236 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.847072 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.847079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.849119 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:25.849135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.851401 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.851410 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:25.851642 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.851778 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 20:28:25.851897 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 20:28:25.852040 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11294,"thread_start_us":11114,"threads_started":1}
I20260806 20:28:25.852545 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:25.831805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.17.109.194:41967)
0806 20:28:25.841637 (+ 9832us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:25.842188 (+ 551us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:25.842953 (+ 765us) server_negotiation.cc:407] Connection header received
0806 20:28:25.843089 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:25.843100 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:25.843401 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:25.843681 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:25.844459 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.846879 (+ 2420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.849320 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:25.850893 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:25.851063 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:25.851231 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:25.851990 (+ 759us) server_negotiation.cc:299] Negotiation successful
0806 20:28:25.852175 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9660,"thread_start_us":9513,"threads_started":1}
I20260806 20:28:25.855994 1010 tablet_service.cc:1467] Processing CreateTablet for tablet 999d1560782342748934c0df1c624311 (DEFAULT_TABLE table=test-table [id=e2639d882a1d4b0790cc56d2c7a58e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:25.856446 875 tablet_service.cc:1467] Processing CreateTablet for tablet 999d1560782342748934c0df1c624311 (DEFAULT_TABLE table=test-table [id=e2639d882a1d4b0790cc56d2c7a58e7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:25.857324 1010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999d1560782342748934c0df1c624311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:25.857767 875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999d1560782342748934c0df1c624311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:25.866408 1098 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Bootstrap starting.
I20260806 20:28:25.871340 1098 tablet_bootstrap.cc:654] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:25.871484 1099 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Bootstrap starting.
I20260806 20:28:25.872982 1098 log.cc:824] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:25.873140 1100 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Bootstrap starting.
I20260806 20:28:25.875362 1099 tablet_bootstrap.cc:654] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:25.876459 1099 log.cc:824] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:25.877002 1100 tablet_bootstrap.cc:654] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:25.877190 1098 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: No bootstrap required, opened a new log
I20260806 20:28:25.877521 1098 ts_tablet_manager.cc:1397] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 20:28:25.878470 1100 log.cc:824] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:25.879350 1099 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: No bootstrap required, opened a new log
I20260806 20:28:25.879724 1099 ts_tablet_manager.cc:1397] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260806 20:28:25.882267 1100 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: No bootstrap required, opened a new log
I20260806 20:28:25.882547 1100 ts_tablet_manager.cc:1397] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 20:28:25.886947 1099 raft_consensus.cc:348] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:25.887341 1099 raft_consensus.cc:374] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:25.887449 1099 raft_consensus.cc:729] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca8f3f0a55fd4d9695c33c1e7f19196f, State: Initialized, Role: FOLLOWER
I20260806 20:28:25.888105 1099 consensus_queue.cc:260] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:25.889228 1082 heartbeater.cc:503] Master 127.17.109.254:34141 was elected leader, sending a full tablet report...
I20260806 20:28:25.890345 1098 raft_consensus.cc:348] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:25.890910 1098 raft_consensus.cc:374] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:25.891067 1098 raft_consensus.cc:729] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65031948d14d47838c64d64d20fff65e, State: Initialized, Role: FOLLOWER
I20260806 20:28:25.891149 1099 ts_tablet_manager.cc:1428] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260806 20:28:25.891870 1098 consensus_queue.cc:260] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:25.894805 1098 ts_tablet_manager.cc:1428] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260806 20:28:25.894486 1100 raft_consensus.cc:348] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:25.894954 1100 raft_consensus.cc:374] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:25.895119 1100 raft_consensus.cc:729] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03df31deb75c41789903dc20b9c965f2, State: Initialized, Role: FOLLOWER
I20260806 20:28:25.895874 1100 consensus_queue.cc:260] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:25.897110 943 heartbeater.cc:503] Master 127.17.109.254:34141 was elected leader, sending a full tablet report...
I20260806 20:28:25.897881 1100 ts_tablet_manager.cc:1428] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260806 20:28:26.019477 1083 tablet.cc:2369] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:28:26.045409 803 tablet.cc:2369] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:28:26.046415 944 tablet.cc:2369] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:28:26.150086 1108 raft_consensus.cc:482] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:26.150447 1108 raft_consensus.cc:504] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:26.152249 1108 leader_election.cc:288] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65031948d14d47838c64d64d20fff65e (127.17.109.193:45529), ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441)
I20260806 20:28:26.159061 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.152691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:38497 (local address 127.17.109.193:45529)
0806 20:28:26.152949 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:26.152956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:26.153697 (+ 741us) server_negotiation.cc:407] Connection header received
0806 20:28:26.154026 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:26.154030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:26.154090 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:26.154191 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:26.154775 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.155484 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.157350 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.158104 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.158288 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:26.158391 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:26.158671 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 20:28:26.158802 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 20:28:26.159214 1109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.152510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.17.109.194:38497)
0806 20:28:26.153555 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.153576 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:26.153844 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:26.154221 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:26.154226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:26.154251 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:26.154617 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:26.154626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.155634 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.155642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:26.157195 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:26.157214 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.158342 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.158348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:26.158509 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:26.158532 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 20:28:26.158638 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 20:28:26.158768 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":30,"thread_start_us":618,"threads_started":1}
I20260806 20:28:26.159286 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.152747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:37383 (local address 127.17.109.195:43441)
0806 20:28:26.152948 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:26.152955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:26.154022 (+ 1067us) server_negotiation.cc:407] Connection header received
0806 20:28:26.154236 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:26.154241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:26.154302 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:26.154440 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:26.155010 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.155803 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.157558 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.158334 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.158533 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:26.158616 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:26.158898 (+ 282us) server_negotiation.cc:299] Negotiation successful
0806 20:28:26.159026 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 20:28:26.159519 1110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.152678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:43441 (local address 127.17.109.194:37383)
0806 20:28:26.153889 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.153903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:26.154071 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:26.154454 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:26.154459 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:26.154485 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:26.154852 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:26.154860 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.155941 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.155947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:26.157399 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:26.157412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.158567 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.158573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:26.158742 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:26.158765 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 20:28:26.158870 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 20:28:26.159017 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"mutex_wait_us":22,"thread_start_us":881,"threads_started":1}
I20260806 20:28:26.160537 753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999d1560782342748934c0df1c624311" candidate_uuid: "03df31deb75c41789903dc20b9c965f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65031948d14d47838c64d64d20fff65e" is_pre_election: true
I20260806 20:28:26.160645 1032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999d1560782342748934c0df1c624311" candidate_uuid: "03df31deb75c41789903dc20b9c965f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" is_pre_election: true
I20260806 20:28:26.161067 753 raft_consensus.cc:2454] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03df31deb75c41789903dc20b9c965f2 in term 0.
I20260806 20:28:26.161163 1032 raft_consensus.cc:2454] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03df31deb75c41789903dc20b9c965f2 in term 0.
I20260806 20:28:26.161816 822 leader_election.cc:302] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03df31deb75c41789903dc20b9c965f2, 65031948d14d47838c64d64d20fff65e, ca8f3f0a55fd4d9695c33c1e7f19196f; no voters:
I20260806 20:28:26.162213 1108 raft_consensus.cc:2781] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:28:26.162388 1108 raft_consensus.cc:482] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:28:26.162496 1108 raft_consensus.cc:3037] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:26.164772 1108 raft_consensus.cc:504] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:26.165380 1108 leader_election.cc:288] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [CANDIDATE]: Term 1 election: Requested vote from peers 65031948d14d47838c64d64d20fff65e (127.17.109.193:45529), ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441)
I20260806 20:28:26.165866 1032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999d1560782342748934c0df1c624311" candidate_uuid: "03df31deb75c41789903dc20b9c965f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f"
I20260806 20:28:26.165845 753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999d1560782342748934c0df1c624311" candidate_uuid: "03df31deb75c41789903dc20b9c965f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65031948d14d47838c64d64d20fff65e"
I20260806 20:28:26.166079 1032 raft_consensus.cc:3037] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:26.166072 753 raft_consensus.cc:3037] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:26.169351 753 raft_consensus.cc:2454] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03df31deb75c41789903dc20b9c965f2 in term 1.
I20260806 20:28:26.169351 1032 raft_consensus.cc:2454] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03df31deb75c41789903dc20b9c965f2 in term 1.
I20260806 20:28:26.169811 822 leader_election.cc:302] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03df31deb75c41789903dc20b9c965f2, 65031948d14d47838c64d64d20fff65e, ca8f3f0a55fd4d9695c33c1e7f19196f; no voters:
I20260806 20:28:26.170145 1108 raft_consensus.cc:2781] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:26.170598 1108 raft_consensus.cc:686] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 1 LEADER]: Becoming Leader. State: Replica: 03df31deb75c41789903dc20b9c965f2, State: Running, Role: LEADER
I20260806 20:28:26.171103 1108 consensus_queue.cc:237] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:26.177212 600 catalog_manager.cc:5495] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03df31deb75c41789903dc20b9c965f2 (127.17.109.194). New cstate: current_term: 1 leader_uuid: "03df31deb75c41789903dc20b9c965f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } health_report { overall_health: HEALTHY } } }
I20260806 20:28:26.209436 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.203787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59822 (local address 127.17.109.194:41967)
0806 20:28:26.203975 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:26.203979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:26.203997 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:28:26.204187 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:26.204190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:26.204274 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:26.204393 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:28:26.204780 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.205512 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.207212 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.207707 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.207832 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:26.208299 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:28:26.208814 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:28:26.208933 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:26.209062 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 20:28:26.209130 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 20:28:26.218155 1032 raft_consensus.cc:1261] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 FOLLOWER]: Refusing update from remote peer 03df31deb75c41789903dc20b9c965f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:26.218173 753 raft_consensus.cc:1261] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Refusing update from remote peer 03df31deb75c41789903dc20b9c965f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:26.219126 1111 consensus_queue.cc:1035] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:26.219595 1108 consensus_queue.cc:1035] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:26.231459 1115 mvcc.cc:205] Tried to move back new op lower bound from 7315653043059957760 to 7315653042881163264. Current Snapshot: MvccSnapshot[applied={T|T < 7315653043059957760}]
I20260806 20:28:26.232249 1114 mvcc.cc:205] Tried to move back new op lower bound from 7315653043059957760 to 7315653042881163264. Current Snapshot: MvccSnapshot[applied={T|T < 7315653043059957760}]
I20260806 20:28:26.243239 600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:28:26.243485 600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:28:26.246914 600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Sending DeleteTablet for 3 replicas of tablet 999d1560782342748934c0df1c624311
I20260806 20:28:26.248060 730 tablet_service.cc:1514] Processing DeleteTablet for tablet 999d1560782342748934c0df1c624311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:26 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:60210
I20260806 20:28:26.248145 1010 tablet_service.cc:1514] Processing DeleteTablet for tablet 999d1560782342748934c0df1c624311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:26 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:38124
I20260806 20:28:26.248734 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 640
I20260806 20:28:26.248682 875 tablet_service.cc:1514] Processing DeleteTablet for tablet 999d1560782342748934c0df1c624311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:26 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:59812
I20260806 20:28:26.270510 1126 tablet_replica.cc:330] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: stopping tablet replica
I20260806 20:28:26.271385 1126 raft_consensus.cc:2229] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:28:26.271931 1126 raft_consensus.cc:2258] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:26.272905 1125 tablet_replica.cc:330] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: stopping tablet replica
I20260806 20:28:26.273321 1125 raft_consensus.cc:2229] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:28:26.273761 1125 raft_consensus.cc:2258] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:26.275353 1126 ts_tablet_manager.cc:1905] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:26.275545 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.249652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60222 (local address 127.17.109.193:45529)
0806 20:28:26.249819 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:26.249826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:26.249987 (+ 161us) server_negotiation.cc:407] Connection header received
0806 20:28:26.250200 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:26.250208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:26.250319 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:26.250417 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:26.252669 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.254170 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.256666 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.263640 (+ 6974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.263781 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:26.267190 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:26.267215 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:26.267239 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:26.267292 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:26.272212 (+ 4920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:26.273097 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:26.273105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:26.273109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:26.273315 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:26.273724 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:26.273730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:26.273733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:26.274247 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:26.274565 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:26.274864 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 20:28:26.275076 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 20:28:26.276464 797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:26.278601 1125 ts_tablet_manager.cc:1905] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:26.279771 1127 tablet_replica.cc:330] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: stopping tablet replica
I20260806 20:28:26.280321 1127 raft_consensus.cc:2229] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:28:26.281185 1127 raft_consensus.cc:2258] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:26.285647 1125 ts_tablet_manager.cc:1918] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:26.285810 1125 log.cc:1196] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/999d1560782342748934c0df1c624311
I20260806 20:28:26.286363 1125 ts_tablet_manager.cc:1939] T 999d1560782342748934c0df1c624311 P ca8f3f0a55fd4d9695c33c1e7f19196f: Deleting consensus metadata
I20260806 20:28:26.288210 585 catalog_manager.cc:4841] TS ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441): tablet 999d1560782342748934c0df1c624311 (table test-table [id=e2639d882a1d4b0790cc56d2c7a58e7b]) successfully deleted
I20260806 20:28:26.291303 1127 ts_tablet_manager.cc:1905] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:26.299876 1127 ts_tablet_manager.cc:1918] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:26.300119 1127 log.cc:1196] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/999d1560782342748934c0df1c624311
I20260806 20:28:26.300845 1127 ts_tablet_manager.cc:1939] T 999d1560782342748934c0df1c624311 P 03df31deb75c41789903dc20b9c965f2: Deleting consensus metadata
I20260806 20:28:26.303306 582 catalog_manager.cc:4841] TS 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967): tablet 999d1560782342748934c0df1c624311 (table test-table [id=e2639d882a1d4b0790cc56d2c7a58e7b]) successfully deleted
I20260806 20:28:26.383090 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 640
W20260806 20:28:26.404814 582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.109.193:45529 (error 108)
I20260806 20:28:26.405078 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 806
I20260806 20:28:26.407460 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.406694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60230)
0806 20:28:26.407186 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.407306 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":271,"threads_started":1}
W20260806 20:28:26.407829 582 catalog_manager.cc:4584] TS 65031948d14d47838c64d64d20fff65e (127.17.109.193:45529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 999d1560782342748934c0df1c624311: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
I20260806 20:28:26.422084 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.406065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.17.109.194:41967)
0806 20:28:26.406256 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:26.406262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:26.406281 (+ 19us) server_negotiation.cc:407] Connection header received
0806 20:28:26.406446 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:26.406451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:26.406572 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:26.406694 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:26.408539 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.409319 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.411313 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.411877 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.411980 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:26.415604 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:26.415626 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:26.415641 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:26.415678 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:26.418992 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:26.419979 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:26.419986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:26.419988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:26.420137 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:26.420549 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:26.420552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:26.420554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:26.420945 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:26.421205 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:26.421544 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 20:28:26.421739 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 20:28:26.422901 938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:26.461748 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.461232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60234)
0806 20:28:26.461462 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.461589 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 20:28:26.462884 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.462531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60250)
0806 20:28:26.462665 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.462754 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 20:28:26.523746 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.523186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60266)
0806 20:28:26.523420 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.523516 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260806 20:28:26.524838 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.524487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60280)
0806 20:28:26.524619 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.524690 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 20:28:26.541224 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 806
I20260806 20:28:26.564373 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 946
I20260806 20:28:26.578840 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.565178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38136 (local address 127.17.109.195:43441)
0806 20:28:26.565385 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:26.565392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:26.565442 (+ 50us) server_negotiation.cc:407] Connection header received
0806 20:28:26.565670 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:26.565680 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:26.565804 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:26.565928 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:26.567433 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.568275 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.569579 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:26.570031 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:26.570171 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:26.573012 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:26.573029 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:26.573045 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:26.573091 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:26.575881 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:26.576634 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:26.576642 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:26.576644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:26.576805 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:26.577208 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:26.577213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:26.577216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:26.577590 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:26.577923 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:26.578236 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 20:28:26.578480 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 20:28:26.579483 1076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:26.605165 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.604636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60294)
0806 20:28:26.604891 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.604993 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260806 20:28:26.606303 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.605878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60300)
0806 20:28:26.606084 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.606156 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 20:28:26.670084 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 946
I20260806 20:28:26.689539 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 563
I20260806 20:28:26.690980 632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:26.811489 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:26.810900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45529 (local address 127.0.0.1:60306)
0806 20:28:26.811114 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:26.811245 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:45529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 20:28:26.811828 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 563
I20260806 20:28:26.837347 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34141
--webserver_interface=127.17.109.254
--webserver_port=43353
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:27.191536 1135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:27.191988 1135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:27.192171 1135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:27.199728 1135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:27.199883 1135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:27.199987 1135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:27.200099 1135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:27.200196 1135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:27.210770 1135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=43353
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:27.212294 1135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:27.213781 1135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:27.226729 1141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:27.226696 1143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:27.226804 1140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:27.227695 1135 server_base.cc:1034] running on GCE node
I20260806 20:28:27.228467 1135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:27.229889 1135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:27.231127 1135 hybrid_clock.cc:648] HybridClock initialized: now 1786048107231074 us; error 58 us; skew 500 ppm
I20260806 20:28:27.231477 1135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:27 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048107, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:27.234683 1135 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:28:27.236214 1135 webserver.cc:460] Webserver started at http://127.17.109.254:43353/ using document root <none> and password file <none>
I20260806 20:28:27.236912 1135 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:27.237062 1135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:27.242547 1135 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 20:28:27.245448 1150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:27.246871 1135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:28:27.247048 1135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "d4f8112ed6524564b85abd04865e4f3f"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "7d4ddf9d8bed820339fd56f71017d557"
server_key_iv: "a1bc55c373dd1adc2921a2d69caf2257"
server_key_version: "encryptionkey@0"
I20260806 20:28:27.248131 1135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:27.264889 1135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:27.269351 1135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:27.269640 1135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:27.289162 1135 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:34141
I20260806 20:28:27.289222 1205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:34141 every 8 connection(s)
I20260806 20:28:27.290776 1135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:28:27.296166 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 1135
I20260806 20:28:27.297073 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:45529
--local_ip_for_outbound_sockets=127.17.109.193
--tserver_master_addrs=127.17.109.254:34141
--webserver_port=41751
--webserver_interface=127.17.109.193
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:28:27.298139 1206 sys_catalog.cc:263] Verifying existing consensus state
I20260806 20:28:27.302115 1206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Bootstrap starting.
I20260806 20:28:27.324132 1206 log.cc:824] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:27.340418 1206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:28:27.340993 1206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f: Bootstrap complete.
I20260806 20:28:27.350253 1206 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:27.351367 1206 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f8112ed6524564b85abd04865e4f3f, State: Initialized, Role: FOLLOWER
I20260806 20:28:27.352144 1206 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:27.352419 1206 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:28:27.352571 1206 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:28:27.352830 1206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 2 FOLLOWER]: Advancing to term 3
I20260806 20:28:27.356842 1206 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:27.357348 1206 leader_election.cc:302] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4f8112ed6524564b85abd04865e4f3f; no voters:
I20260806 20:28:27.358062 1206 leader_election.cc:288] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 20:28:27.358212 1210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 3 FOLLOWER]: Leader election won for term 3
I20260806 20:28:27.359599 1210 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [term 3 LEADER]: Becoming Leader. State: Replica: d4f8112ed6524564b85abd04865e4f3f, State: Running, Role: LEADER
I20260806 20:28:27.360198 1210 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } }
I20260806 20:28:27.360503 1206 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:27.362874 1211 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d4f8112ed6524564b85abd04865e4f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } } }
I20260806 20:28:27.363302 1211 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:27.367162 1212 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4f8112ed6524564b85abd04865e4f3f. Latest consensus state: current_term: 3 leader_uuid: "d4f8112ed6524564b85abd04865e4f3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f8112ed6524564b85abd04865e4f3f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34141 } } }
I20260806 20:28:27.367550 1212 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4f8112ed6524564b85abd04865e4f3f [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:27.369333 1216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:28:27.378516 1216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:28:27.380235 1216 catalog_manager.cc:1259] Loaded cluster ID: 6bea52ace37d4f508c40c163459470b3
I20260806 20:28:27.380403 1216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 20:28:27.382594 1216 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1efdc939ad google::LogMessage::Fail() at ??:0
@ 0x7f1efdc9688a google::LogMessage::SendToLog() at ??:0
@ 0x7f1efdc934de google::LogMessage::Flush() at ??:0
@ 0x7f1efdc94a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260806 20:28:27.391639 1229 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f1f136ca512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f1f136c7976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1f1374f610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f1f1374f439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1f008df345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1f008e291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f1f008e2799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1f008bdeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1f03b976db start_thread at ??:0
@ 0x7f1efcb2d61f clone at ??:0
W20260806 20:28:27.727545 1208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:27.727890 1208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:27.727993 1208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:27.735240 1208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:27.735343 1208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:27.735461 1208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:28:27.745381 1208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:45529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=41751
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:27.746750 1208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:27.748198 1208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:27.762501 1250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:27.762480 1252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:27.762838 1249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:27.763459 1208 server_base.cc:1034] running on GCE node
I20260806 20:28:27.764119 1208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:27.765066 1208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:27.766300 1208 hybrid_clock.cc:648] HybridClock initialized: now 1786048107766262 us; error 56 us; skew 500 ppm
I20260806 20:28:27.766619 1208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:27 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048107, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:27.769791 1208 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:28:27.771379 1208 webserver.cc:460] Webserver started at http://127.17.109.193:41751/ using document root <none> and password file <none>
I20260806 20:28:27.772150 1208 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:27.772308 1208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:27.777717 1208 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 20:28:27.780874 1259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:27.782233 1208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:28:27.782415 1208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "65031948d14d47838c64d64d20fff65e"
format_stamp: "Formatted at 2026-08-06 20:28:21 on dist-test-slave-6k22"
server_key: "12146bf5727e2056bf99a28fe94cde80"
server_key_iv: "3f253c12271de42e592fa672b23c74ee"
server_key_version: "encryptionkey@0"
I20260806 20:28:27.783396 1208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:27.821739 1208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:27.826203 1208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:27.826534 1208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:27.827656 1208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:27.829573 1266 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 20:28:27.835345 1208 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 20:28:27.835507 1208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260806 20:28:27.835672 1208 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 20:28:27.838778 1208 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 20:28:27.838896 1208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260806 20:28:27.839272 1266 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Bootstrap starting.
I20260806 20:28:27.848981 1266 log.cc:824] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:27.868291 1266 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:28:27.869204 1266 tablet_bootstrap.cc:492] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Bootstrap complete.
I20260806 20:28:27.869791 1266 ts_tablet_manager.cc:1397] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Time spent bootstrapping tablet: real 0.031s user 0.024s sys 0.004s
I20260806 20:28:27.878645 1208 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:45529
I20260806 20:28:27.878690 1384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:45529 every 8 connection(s)
I20260806 20:28:27.880364 1208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:28:27.882817 1266 raft_consensus.cc:348] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:27.884330 1266 raft_consensus.cc:729] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65031948d14d47838c64d64d20fff65e, State: Initialized, Role: FOLLOWER
I20260806 20:28:27.885046 1266 consensus_queue.cc:260] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:27.886067 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:27.883492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:44505)
0806 20:28:27.884503 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:27.884859 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":395,"threads_started":1}
I20260806 20:28:27.886716 1266 ts_tablet_manager.cc:1428] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e: Time spent starting tablet: real 0.017s user 0.011s sys 0.007s
W20260806 20:28:27.886822 1385 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:34141 (0 consecutive failures): Network error: Failed to ping master at 127.17.109.254:34141: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
I20260806 20:28:27.887691 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:27.887322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:55293)
0806 20:28:27.887467 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:27.887548 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 20:28:27.888813 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:27.888279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:47267)
0806 20:28:27.888439 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:27.888605 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 20:28:27.889071 1385 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 20:28:27.889288 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 1208
I20260806 20:28:27.889986 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:41967
--local_ip_for_outbound_sockets=127.17.109.194
--tserver_master_addrs=127.17.109.254:34141
--webserver_port=43231
--webserver_interface=127.17.109.194
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:28.231335 1389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:28.231642 1389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:28.231745 1389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:28.238886 1389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:28.238997 1389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:28.239112 1389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:28:28.248768 1389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:41967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=43231
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:28.249996 1389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:28.251242 1389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:28.264621 1396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:28.264644 1395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:28.264763 1398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:28.265420 1389 server_base.cc:1034] running on GCE node
I20260806 20:28:28.266058 1389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:28.267014 1389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:28.268244 1389 hybrid_clock.cc:648] HybridClock initialized: now 1786048108268188 us; error 69 us; skew 500 ppm
I20260806 20:28:28.268561 1389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:28 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048108, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:28.271762 1389 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:28:28.273190 1389 webserver.cc:460] Webserver started at http://127.17.109.194:43231/ using document root <none> and password file <none>
I20260806 20:28:28.273882 1389 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:28.274034 1389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:28.279454 1389 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:28:28.282264 1407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:28.283828 1389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:28:28.284013 1389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "03df31deb75c41789903dc20b9c965f2"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "40ff8ff92eb2f4f47316f41a1169be13"
server_key_iv: "ff6aecdd7c185b33a39a248e74ba95d8"
server_key_version: "encryptionkey@0"
I20260806 20:28:28.285001 1389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:28.304330 1389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:28.308477 1389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:28.308768 1389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:28.309798 1389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:28.311168 1389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:28.311316 1389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:28.311452 1389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:28.311544 1389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:28.346670 1389 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:41967
I20260806 20:28:28.346769 1535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:41967 every 8 connection(s)
I20260806 20:28:28.348210 1389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:28:28.349296 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 1389
I20260806 20:28:28.350099 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:43441
--local_ip_for_outbound_sockets=127.17.109.195
--tserver_master_addrs=127.17.109.254:34141
--webserver_port=44001
--webserver_interface=127.17.109.195
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:45331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:28:28.353865 1538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:28.351544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:34431)
0806 20:28:28.352444 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:28.352704 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":298,"threads_started":1}
W20260806 20:28:28.354622 1536 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:34141 (0 consecutive failures): Network error: Failed to ping master at 127.17.109.254:34141: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
I20260806 20:28:28.355621 1538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:28.355152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:49885)
0806 20:28:28.355343 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:28.355455 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 20:28:28.356619 1538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:28.356198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:34299)
0806 20:28:28.356368 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:28.356467 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 20:28:28.356961 1536 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 20:28:28.685345 1539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:28.685683 1539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:28.685861 1539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:28.693475 1539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:28.693642 1539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:28.693825 1539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:28:28.703883 1539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:45331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:43441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=44001
--tserver_master_addrs=127.17.109.254:34141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:28.705358 1539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:28.706830 1539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:28.720669 1547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:28.720741 1544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:28.720649 1545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:28.721586 1539 server_base.cc:1034] running on GCE node
I20260806 20:28:28.722271 1539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:28.723305 1539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:28.724562 1539 hybrid_clock.cc:648] HybridClock initialized: now 1786048108724499 us; error 52 us; skew 500 ppm
I20260806 20:28:28.724931 1539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:28 dist-test-slave-6k22 krb5kdc[32449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048108, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:28.728277 1539 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:28:28.729873 1539 webserver.cc:460] Webserver started at http://127.17.109.195:44001/ using document root <none> and password file <none>
I20260806 20:28:28.730592 1539 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:28.730751 1539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:28.736249 1539 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:28:28.739097 1554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:28.740525 1539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:28:28.740717 1539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f"
format_stamp: "Formatted at 2026-08-06 20:28:22 on dist-test-slave-6k22"
server_key: "5dc8d2dfba966386f4017c07df73279b"
server_key_iv: "772204d61a3dbeb95d1b838be5bdac1c"
server_key_version: "encryptionkey@0"
I20260806 20:28:28.741709 1539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:28.767692 1539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:28.772002 1539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:28.772296 1539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:28.773391 1539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:28.774727 1539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:28.774864 1539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:28.775002 1539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:28.775094 1539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:28.810799 1539 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:43441
I20260806 20:28:28.810858 1679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:43441 every 8 connection(s)
I20260806 20:28:28.812366 1539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:28:28.817303 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:28.815313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:45637)
0806 20:28:28.816153 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:28.816399 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":334,"threads_started":1}
W20260806 20:28:28.817994 1680 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:34141 (0 consecutive failures): Network error: Failed to ping master at 127.17.109.254:34141: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
I20260806 20:28:28.818784 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:28.818462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:57899)
0806 20:28:28.818610 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:28.818681 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 20:28:28.819749 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:28.819322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:46357)
0806 20:28:28.819474 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:28.819613 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 20:28:28.819983 1680 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 20:28:28.820933 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 1539
I20260806 20:28:28.890465 1684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:28.889626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:34827)
0806 20:28:28.890154 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:28.890273 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":274,"threads_started":1}
I20260806 20:28:29.358347 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:29.357512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:59265)
0806 20:28:29.357981 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:29.358110 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":229,"threads_started":1}
I20260806 20:28:29.402673 1686 raft_consensus.cc:482] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:29.402980 1686 raft_consensus.cc:504] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:29.404636 1686 leader_election.cc:288] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441), 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967)
Aug 06 20:28:29 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048107, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
Aug 06 20:28:29 dist-test-slave-6k22 krb5kdc[32449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048107, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:28:29.428452 1692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:29.405279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:38219 (local address 127.17.109.194:41967)
0806 20:28:29.407449 (+ 2170us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:29.407470 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:29.408069 (+ 599us) server_negotiation.cc:407] Connection header received
0806 20:28:29.409731 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:29.409742 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:29.410188 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:29.410542 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:29.412072 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:29.413506 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:29.415263 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:29.415980 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:29.416169 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:29.420507 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:29.420513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:29.420524 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:29.420560 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:29.423541 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:29.424600 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:29.424607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:29.424613 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:29.424831 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:29.425465 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:29.425471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:29.425474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:29.426205 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:29.426404 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:29.427539 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0806 20:28:29.428023 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1914,"thread_start_us":1707,"threads_started":1}
I20260806 20:28:29.428453 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:29.405126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:53169 (local address 127.17.109.195:43441)
0806 20:28:29.407667 (+ 2541us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:29.407678 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:29.407711 (+ 33us) server_negotiation.cc:407] Connection header received
0806 20:28:29.409117 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:29.409126 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:29.409514 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:29.409782 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:29.412092 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:29.413628 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:29.415299 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:29.415989 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:29.416202 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:29.420222 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:29.420246 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:29.420273 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:29.420334 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:29.423409 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:29.424591 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:29.424600 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:29.424608 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:29.424831 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:29.425465 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:29.425471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:29.425474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:29.426205 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:29.426404 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:29.427539 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0806 20:28:29.428023 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2368,"thread_start_us":2260,"threads_started":1}
I20260806 20:28:29.428442 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:29.404897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:41967 (local address 127.17.109.193:38219)
0806 20:28:29.407948 (+ 3051us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:29.407963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:29.408106 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:29.410503 (+ 2397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:29.410506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:29.411045 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:29.411875 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:29.411896 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:29.413691 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:29.413699 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:29.415116 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:29.415129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:29.416197 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:29.416204 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:29.416490 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:29.417293 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:29.417317 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:29.420251 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:29.423701 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:29.423707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:29.423713 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:29.424431 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:29.424973 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:29.424978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:29.424982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:29.425322 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:29.426368 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:29.426377 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:29.426921 (+ 544us) client_negotiation.cc:769] Sending connection context
0806 20:28:29.427254 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 20:28:29.427882 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2950,"mutex_wait_us":262,"spinlock_wait_cycles":26624,"thread_start_us":2788,"threads_started":1}
I20260806 20:28:29.428465 1687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:29.404872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:43441 (local address 127.17.109.193:53169)
0806 20:28:29.405636 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:29.405685 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:29.406373 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:29.409976 (+ 3603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:29.409987 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:29.411035 (+ 1048us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:29.411875 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:29.411897 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:29.413806 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:29.413811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:29.415122 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:29.415135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:29.416217 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:29.416223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:29.416488 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:29.417353 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:29.417370 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:29.419904 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:29.423601 (+ 3697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:29.423612 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:29.423627 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:29.424433 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:29.425001 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:29.425005 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:29.425008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:29.425322 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:29.426368 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:29.426377 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:29.426921 (+ 544us) client_negotiation.cc:769] Sending connection context
0806 20:28:29.427254 (+ 333us) client_negotiation.cc:240] Negotiation successful
0806 20:28:29.428001 (+ 747us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":411,"spinlock_wait_cycles":34432,"thread_start_us":574,"threads_started":1}
W20260806 20:28:29.432178 1260 leader_election.cc:341] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967): Not found: Tablet not found: 999d1560782342748934c0df1c624311
W20260806 20:28:29.432495 1263 leader_election.cc:341] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441): Not found: Tablet not found: 999d1560782342748934c0df1c624311
I20260806 20:28:29.432626 1263 leader_election.cc:302] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65031948d14d47838c64d64d20fff65e; no voters: 03df31deb75c41789903dc20b9c965f2, ca8f3f0a55fd4d9695c33c1e7f19196f
I20260806 20:28:29.433000 1686 raft_consensus.cc:2726] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 20:28:29.821460 1693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:29.820570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:40575)
0806 20:28:29.821060 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:29.821230 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":240,"threads_started":1}
I20260806 20:28:29.891866 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:29.891389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:33843)
0806 20:28:29.891603 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:29.891707 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 20:28:30.360122 1694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:30.359305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:51401)
0806 20:28:30.359856 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:30.359948 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":306,"threads_started":1}
I20260806 20:28:30.823132 1695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:30.822346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:43669)
0806 20:28:30.822884 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:30.822972 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":326,"threads_started":1}
I20260806 20:28:30.893445 1696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:30.892714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:56039)
0806 20:28:30.893171 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:30.893254 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":233,"threads_started":1}
I20260806 20:28:31.303824 1697 raft_consensus.cc:482] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:31.304056 1697 raft_consensus.cc:504] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:31.304829 1697 leader_election.cc:288] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441), 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967)
W20260806 20:28:31.305867 1263 leader_election.cc:341] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441): Not found: Tablet not found: 999d1560782342748934c0df1c624311
W20260806 20:28:31.306048 1260 leader_election.cc:341] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967): Not found: Tablet not found: 999d1560782342748934c0df1c624311
I20260806 20:28:31.306162 1260 leader_election.cc:302] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65031948d14d47838c64d64d20fff65e; no voters: 03df31deb75c41789903dc20b9c965f2, ca8f3f0a55fd4d9695c33c1e7f19196f
I20260806 20:28:31.306417 1697 raft_consensus.cc:2726] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 20:28:31.361698 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:31.360993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:36065)
0806 20:28:31.361434 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:31.361521 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":237,"threads_started":1}
I20260806 20:28:31.824864 1699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:31.824113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:45833)
0806 20:28:31.824620 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:31.824699 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":306,"threads_started":1}
I20260806 20:28:31.895025 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:31.894324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:34439)
0806 20:28:31.894754 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:31.894841 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":234,"threads_started":1}
I20260806 20:28:32.363407 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:32.362636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:59309)
0806 20:28:32.363125 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:32.363218 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":237,"threads_started":1}
I20260806 20:28:32.826707 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:32.825899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:51531)
0806 20:28:32.826418 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:32.826516 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":262,"threads_started":1}
I20260806 20:28:32.897197 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:32.896297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:41001)
0806 20:28:32.896900 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:32.896994 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":242,"threads_started":1}
I20260806 20:28:33.365216 1704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:33.364466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:52561)
0806 20:28:33.364945 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:33.365037 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":265,"threads_started":1}
I20260806 20:28:33.828600 1705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:33.827871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:53129)
0806 20:28:33.828350 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:33.828433 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":251,"threads_started":1}
I20260806 20:28:33.898866 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:33.898134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:35347)
0806 20:28:33.898584 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:33.898667 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":257,"threads_started":1}
I20260806 20:28:34.166313 1707 raft_consensus.cc:482] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:34.166514 1707 raft_consensus.cc:504] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:34.167196 1707 leader_election.cc:288] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441), 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967)
W20260806 20:28:34.168161 1260 leader_election.cc:341] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967): Not found: Tablet not found: 999d1560782342748934c0df1c624311
W20260806 20:28:34.168310 1263 leader_election.cc:341] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441): Not found: Tablet not found: 999d1560782342748934c0df1c624311
I20260806 20:28:34.168485 1263 leader_election.cc:302] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65031948d14d47838c64d64d20fff65e; no voters: 03df31deb75c41789903dc20b9c965f2, ca8f3f0a55fd4d9695c33c1e7f19196f
I20260806 20:28:34.168740 1707 raft_consensus.cc:2726] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 20:28:34.367110 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:34.366272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:34361)
0806 20:28:34.366830 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:34.366919 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":262,"threads_started":1}
I20260806 20:28:34.830605 1710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:34.829677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:59619)
0806 20:28:34.830271 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:34.830358 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":350,"threads_started":1}
I20260806 20:28:34.900758 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:34.899926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:59319)
0806 20:28:34.900440 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:34.900542 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":248,"threads_started":1}
I20260806 20:28:35.369119 1712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:35.368168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:37471)
0806 20:28:35.368847 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:35.368939 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":336,"threads_started":1}
I20260806 20:28:35.832674 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:35.831756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:37863)
0806 20:28:35.832382 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:35.832480 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":274,"threads_started":1}
I20260806 20:28:35.902732 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:35.901918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:58441)
0806 20:28:35.902455 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:35.902558 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":282,"threads_started":1}
I20260806 20:28:36.371001 1715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:36.370183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:53377)
0806 20:28:36.370736 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:36.370823 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":285,"threads_started":1}
I20260806 20:28:36.834393 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:36.833696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:43371)
0806 20:28:36.834121 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:36.834206 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":227,"threads_started":1}
I20260806 20:28:36.904520 1717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:36.903717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:35311)
0806 20:28:36.904259 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:36.904347 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":308,"threads_started":1}
I20260806 20:28:37.372838 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:37.372004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:59459)
0806 20:28:37.372494 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:37.372618 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":237,"threads_started":1}
I20260806 20:28:37.836158 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:37.835363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:41779)
0806 20:28:37.835912 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:37.835999 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":314,"threads_started":1}
I20260806 20:28:37.906208 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:37.905504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:35725)
0806 20:28:37.905946 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:37.906032 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":245,"threads_started":1}
I20260806 20:28:37.924382 1722 raft_consensus.cc:482] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:37.924618 1722 raft_consensus.cc:504] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65031948d14d47838c64d64d20fff65e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45529 } } peers { permanent_uuid: "ca8f3f0a55fd4d9695c33c1e7f19196f" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43441 } } peers { permanent_uuid: "03df31deb75c41789903dc20b9c965f2" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 41967 } }
I20260806 20:28:37.925248 1722 leader_election.cc:288] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441), 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967)
W20260806 20:28:37.926198 1263 leader_election.cc:341] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca8f3f0a55fd4d9695c33c1e7f19196f (127.17.109.195:43441): Not found: Tablet not found: 999d1560782342748934c0df1c624311
W20260806 20:28:37.926442 1260 leader_election.cc:341] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 03df31deb75c41789903dc20b9c965f2 (127.17.109.194:41967): Not found: Tablet not found: 999d1560782342748934c0df1c624311
I20260806 20:28:37.926617 1260 leader_election.cc:302] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65031948d14d47838c64d64d20fff65e; no voters: 03df31deb75c41789903dc20b9c965f2, ca8f3f0a55fd4d9695c33c1e7f19196f
I20260806 20:28:37.927023 1722 raft_consensus.cc:2726] T 999d1560782342748934c0df1c624311 P 65031948d14d47838c64d64d20fff65e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 20:28:38.374543 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:38.373825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:45849)
0806 20:28:38.374260 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:38.374342 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":219,"threads_started":1}
I20260806 20:28:38.837917 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:38.837177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:55745)
0806 20:28:38.837641 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:38.837729 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":257,"threads_started":1}
I20260806 20:28:38.907999 1725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:38.907132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:43003)
0806 20:28:38.907684 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:38.907791 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":313,"threads_started":1}
I20260806 20:28:39.376551 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:39.375760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:42713)
0806 20:28:39.376274 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:39.376362 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":254,"threads_started":1}
I20260806 20:28:39.839702 1728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:39.838830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:40937)
0806 20:28:39.839354 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:39.839448 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":268,"threads_started":1}
I20260806 20:28:39.909689 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:39.908940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:54851)
0806 20:28:39.909413 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:39.909499 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":263,"threads_started":1}
I20260806 20:28:40.378558 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:40.377682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:51097)
0806 20:28:40.378227 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:40.378324 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":274,"threads_started":1}
I20260806 20:28:40.841509 1731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:40.840726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:40603)
0806 20:28:40.841226 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:40.841332 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":271,"threads_started":1}
I20260806 20:28:40.911420 1732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:40.910630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:55161)
0806 20:28:40.911131 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:40.911218 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":287,"threads_started":1}
I20260806 20:28:41.380501 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:41.379711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:46807)
0806 20:28:41.380195 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:41.380327 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":249,"threads_started":1}
I20260806 20:28:41.843425 1734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:41.842574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:34627)
0806 20:28:41.843123 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:41.843230 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":364,"threads_started":1}
I20260806 20:28:41.913173 1735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:41.912425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:53999)
0806 20:28:41.912880 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:41.912974 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":246,"threads_started":1}
I20260806 20:28:42.382266 1736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:42.381468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:57083)
0806 20:28:42.381972 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:42.382075 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":274,"threads_started":1}
I20260806 20:28:42.845276 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:42.844409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:56913)
0806 20:28:42.844901 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:42.844993 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":239,"threads_started":1}
I20260806 20:28:42.915264 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:42.914392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:45347)
0806 20:28:42.914941 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:42.915053 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":244,"threads_started":1}
I20260806 20:28:43.384290 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:43.383466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.194:37277)
0806 20:28:43.384025 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:43.384123 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":329,"threads_started":1}
I20260806 20:28:43.827098 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 1208
I20260806 20:28:43.847374 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:43.846522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.195:34059)
0806 20:28:43.847080 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:43.847202 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":294,"threads_started":1}
I20260806 20:28:43.852538 1740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:43.828293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.17.109.193:45529)
0806 20:28:43.828895 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:43.828905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:43.828937 (+ 32us) server_negotiation.cc:407] Connection header received
0806 20:28:43.829011 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:43.829020 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:43.829334 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:43.829528 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:43.831530 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:43.832559 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:43.833778 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:43.834339 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:43.834442 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:43.846626 (+ 12184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:43.846643 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:43.846660 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:43.846698 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:43.849786 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:43.850490 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:43.850497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:43.850499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:43.850628 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:43.850991 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:43.850995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:43.850997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:43.851352 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:43.851656 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:43.851934 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 20:28:43.852150 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":267,"threads_started":1}
I20260806 20:28:43.853824 1379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:43.945188 1744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:43.944074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34141 (local address 127.17.109.193:54711)
0806 20:28:43.944648 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:43.944823 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:34141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":264,"threads_started":1}
I20260806 20:28:43.945338 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 1208
I20260806 20:28:43.967694 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 1389
I20260806 20:28:43.981307 1745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:43.968523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.17.109.194:41967)
0806 20:28:43.969056 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:43.969061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:43.969078 (+ 17us) server_negotiation.cc:407] Connection header received
0806 20:28:43.969135 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:43.969138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:43.969198 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:43.969290 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:43.970667 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:43.971454 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:43.972741 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:43.973185 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:43.973282 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:43.976130 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:43.976135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:43.976138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:43.976172 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:43.978715 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:43.979457 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:43.979461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:43.979463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:43.979674 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:43.980014 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:43.980017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:43.980019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:43.980280 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:43.980491 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:43.980715 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 20:28:43.980907 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":313,"threads_started":1}
I20260806 20:28:43.982077 1530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:44.072304 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 1389
I20260806 20:28:44.094849 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 1539
I20260806 20:28:44.109115 1747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:44.095915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.17.109.195:43441)
0806 20:28:44.096562 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:44.096566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:44.096584 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:28:44.096640 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:44.096643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:44.096712 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:44.096823 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:44.098288 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:44.099077 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:44.100581 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:44.101038 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:44.101136 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:44.103805 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:44.103810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:44.103813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:44.103844 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:44.106631 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:44.107320 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:44.107324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:44.107326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:44.107453 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:44.107865 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:44.107869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:44.107871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:44.108158 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:44.108374 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:44.108565 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 20:28:44.108730 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":363,"threads_started":1}
I20260806 20:28:44.109841 1675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:44.197268 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 1539
2026-08-06T20:28:44Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25033 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:28:44 dist-test-slave-6k22 krb5kdc[1751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:28:44 dist-test-slave-6k22 krb5kdc[1751](info): set up 2 sockets
Aug 06 20:28:44 dist-test-slave-6k22 krb5kdc[1751](info): commencing operation
krb5kdc: starting...
W20260806 20:28:45.386498 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:28:45 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:28:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:28:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:28:45.698719 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46883
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:46.049844 1765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:46.050140 1765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:46.050298 1765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:46.057614 1765 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 20:28:46.057729 1765 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 20:28:46.057818 1765 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 20:28:46.057898 1765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:28:46.057973 1765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:46.058099 1765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:46.058183 1765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:28:46.058272 1765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:28:46.068468 1765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:46.069903 1765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:46.071305 1765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:46.083958 1770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:46.084052 1773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:46.084077 1771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:46.084736 1765 server_base.cc:1034] running on GCE node
I20260806 20:28:46.085518 1765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:46.087102 1765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:46.088377 1765 hybrid_clock.cc:648] HybridClock initialized: now 1786048126088265 us; error 71 us; skew 500 ppm
I20260806 20:28:46.088693 1765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:46 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048126, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:46.092612 1765 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:28:46.095029 1765 webserver.cc:460] Webserver started at http://127.17.109.254:42853/ using document root <none> and password file <none>
I20260806 20:28:46.095880 1765 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:46.095974 1765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:46.096329 1765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:46.099502 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "a1f104883451419d8828e780a72e58bc"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "5bc853282a857b4269b0d8d62b47a62f"
server_key_iv: "263bb76a10a91476f179bb301f672efa"
server_key_version: "encryptionkey@0"
I20260806 20:28:46.100394 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "a1f104883451419d8828e780a72e58bc"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "5bc853282a857b4269b0d8d62b47a62f"
server_key_iv: "263bb76a10a91476f179bb301f672efa"
server_key_version: "encryptionkey@0"
I20260806 20:28:46.105839 1765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:28:46.109396 1780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:46.110837 1765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:28:46.111060 1765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "a1f104883451419d8828e780a72e58bc"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "5bc853282a857b4269b0d8d62b47a62f"
server_key_iv: "263bb76a10a91476f179bb301f672efa"
server_key_version: "encryptionkey@0"
I20260806 20:28:46.111337 1765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:46.124477 1765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:46.128650 1765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:46.128937 1765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:46.148077 1765 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:46883
I20260806 20:28:46.148102 1837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:46883 every 8 connection(s)
I20260806 20:28:46.149601 1765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:28:46.153901 1838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:46.158771 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 1765
I20260806 20:28:46.159031 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:28:46.159792 17847 external_mini_cluster.cc:1428] Setting key 71e2790200af5168439af2fc016d8c05
I20260806 20:28:46.164947 1838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Bootstrap starting.
I20260806 20:28:46.168838 1838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:46.170131 1838 log.cc:824] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:46.174103 1838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: No bootstrap required, opened a new log
Aug 06 20:28:46 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:46.185105 1838 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:28:46.185614 1838 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:46.185752 1838 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f104883451419d8828e780a72e58bc, State: Initialized, Role: FOLLOWER
I20260806 20:28:46.186420 1838 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:28:46.186584 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:46.161903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60930 (local address 127.17.109.254:46883)
0806 20:28:46.162801 (+ 898us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:46.162822 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:46.162865 (+ 43us) server_negotiation.cc:407] Connection header received
0806 20:28:46.164607 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:46.164629 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:46.165029 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:46.165393 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:46.167390 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:46.168757 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:46.170660 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:46.171826 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:46.171956 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:46.177561 (+ 5605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:46.177575 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:46.177603 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:46.177663 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:46.182029 (+ 4366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:46.182961 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:46.182970 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:46.182978 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:46.183163 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:46.183601 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:46.183606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:46.183610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:46.184157 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:46.184495 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:46.184927 (+ 432us) server_negotiation.cc:299] Negotiation successful
0806 20:28:46.185368 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":418,"threads_started":1}
I20260806 20:28:46.186697 1838 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:28:46.186878 1838 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:28:46.187093 1838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:46.189878 1838 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:28:46.190338 1838 leader_election.cc:302] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1f104883451419d8828e780a72e58bc; no voters:
I20260806 20:28:46.190894 1838 leader_election.cc:288] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:28:46.191030 1843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:46.191843 1843 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 1 LEADER]: Becoming Leader. State: Replica: a1f104883451419d8828e780a72e58bc, State: Running, Role: LEADER
I20260806 20:28:46.192404 1843 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:28:46.193027 1838 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:28:46.196033 1844 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1f104883451419d8828e780a72e58bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } } }
I20260806 20:28:46.196424 1844 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:46.196682 1845 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1f104883451419d8828e780a72e58bc. Latest consensus state: current_term: 1 leader_uuid: "a1f104883451419d8828e780a72e58bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } } }
I20260806 20:28:46.197106 1845 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: This master's current role is: LEADER
I20260806 20:28:46.204555 1849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:28:46.210187 1849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:28:46.219970 1849 catalog_manager.cc:1347] Generated new cluster ID: 0e7bc51c390243099d1947ecb3c0dfa6
I20260806 20:28:46.220101 1849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:28:46.243459 1849 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:28:46.245150 1849 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:28:46.252980 1849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Generated new TSK 0
I20260806 20:28:46.253790 1849 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:28:46.331308 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46883
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:46.681362 1865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:46.681713 1865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:46.681865 1865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:46.689477 1865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:46.689638 1865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:46.689803 1865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:28:46.700461 1865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:46.701834 1865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:46.703190 1865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:46.716831 1871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:46.716893 1870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:46.716779 1873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:46.721179 1865 server_base.cc:1034] running on GCE node
I20260806 20:28:46.721872 1865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:46.722916 1865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:46.724143 1865 hybrid_clock.cc:648] HybridClock initialized: now 1786048126724097 us; error 40 us; skew 500 ppm
I20260806 20:28:46.724465 1865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:46 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048126, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:46.727716 1865 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:28:46.729275 1865 webserver.cc:460] Webserver started at http://127.17.109.193:39391/ using document root <none> and password file <none>
I20260806 20:28:46.730010 1865 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:46.730147 1865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:46.730471 1865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:46.733520 1865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "b3dbc7b0bc8d466999992f2942f1d553"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "f57e35a9eb53e5747f08b8bfc6cc7f3c"
server_key_iv: "0cf1a719207111e08d2476ec23bca448"
server_key_version: "encryptionkey@0"
I20260806 20:28:46.734293 1865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "b3dbc7b0bc8d466999992f2942f1d553"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "f57e35a9eb53e5747f08b8bfc6cc7f3c"
server_key_iv: "0cf1a719207111e08d2476ec23bca448"
server_key_version: "encryptionkey@0"
I20260806 20:28:46.739706 1865 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 20:28:46.743257 1880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:46.744760 1865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:28:46.744958 1865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "b3dbc7b0bc8d466999992f2942f1d553"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "f57e35a9eb53e5747f08b8bfc6cc7f3c"
server_key_iv: "0cf1a719207111e08d2476ec23bca448"
server_key_version: "encryptionkey@0"
I20260806 20:28:46.745173 1865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:46.770565 1865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:46.775424 1865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:46.775763 1865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:46.776754 1865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:46.778108 1865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:46.778182 1865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:46.778342 1865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:46.778430 1865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:46.812273 1865 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:37011
I20260806 20:28:46.812320 2002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:37011 every 8 connection(s)
I20260806 20:28:46.813563 1865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:28:46.814618 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 1865
I20260806 20:28:46.814844 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:28:46.815752 17847 external_mini_cluster.cc:1428] Setting key df541f83c179cf5e55229295ece65516
Aug 06 20:28:46 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048126, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:46.836229 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:46.816437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:43963 (local address 127.17.109.254:46883)
0806 20:28:46.816946 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:46.816952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:46.818320 (+ 1368us) server_negotiation.cc:407] Connection header received
0806 20:28:46.820315 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:46.820324 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:46.820422 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:46.820547 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:46.822020 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:46.823169 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:46.824849 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:46.825554 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:46.825710 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:46.829894 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:46.829900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:46.829903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:46.829938 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:46.832600 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:46.833631 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:46.833635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:46.833638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:46.833765 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:46.834341 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:46.834344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:46.834346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:46.834615 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:46.834729 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:46.835755 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0806 20:28:46.835952 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":265,"threads_started":1}
I20260806 20:28:46.837984 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:46.817089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:43963)
0806 20:28:46.818136 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:46.818190 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:46.820050 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:46.820747 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:46.820755 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:46.821283 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:46.821847 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:46.821864 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:46.823342 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:46.823346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:46.824680 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:46.824693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:46.825813 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:46.825819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:46.826044 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:46.826861 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:46.826885 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:46.829621 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:46.832784 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:46.832797 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:46.832817 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:46.833488 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:46.833904 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:46.833912 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:46.833918 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:46.834206 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:46.834748 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:46.834759 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:46.835251 (+ 492us) client_negotiation.cc:769] Sending connection context
0806 20:28:46.835680 (+ 429us) client_negotiation.cc:240] Negotiation successful
0806 20:28:46.836363 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":577,"threads_started":1}
I20260806 20:28:46.839776 2003 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46883
I20260806 20:28:46.840143 2003 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:46.840914 2003 heartbeater.cc:511] Master 127.17.109.254:46883 requested a full tablet report, sending...
I20260806 20:28:46.843163 1798 ts_manager.cc:194] Registered new tserver with Master: b3dbc7b0bc8d466999992f2942f1d553 (127.17.109.193:37011)
I20260806 20:28:46.844748 1798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:43963
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:28:46.884879 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46883
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:47.245415 2009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:47.245787 2009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:47.245956 2009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:47.253669 2009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:47.253850 2009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:47.254032 2009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:28:47.264364 2009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:47.265842 2009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:47.267221 2009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:47.281075 2014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:47.280987 2017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:47.281036 2015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:47.281874 2009 server_base.cc:1034] running on GCE node
I20260806 20:28:47.282497 2009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:47.283470 2009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:47.284677 2009 hybrid_clock.cc:648] HybridClock initialized: now 1786048127284624 us; error 47 us; skew 500 ppm
I20260806 20:28:47.284976 2009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:28:47 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048127, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:47.288348 2009 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:28:47.289798 2009 webserver.cc:460] Webserver started at http://127.17.109.194:34881/ using document root <none> and password file <none>
I20260806 20:28:47.290529 2009 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:47.290664 2009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:47.290992 2009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:47.294329 2009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "baeeb821d49c4305856834c1a1c65cc4"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "8d39b7904361fbcc86caf5335bf095f9"
server_key_iv: "cdc46a23e071b1f8a76535ff5174d16f"
server_key_version: "encryptionkey@0"
I20260806 20:28:47.295123 2009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "baeeb821d49c4305856834c1a1c65cc4"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "8d39b7904361fbcc86caf5335bf095f9"
server_key_iv: "cdc46a23e071b1f8a76535ff5174d16f"
server_key_version: "encryptionkey@0"
I20260806 20:28:47.300660 2009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 20:28:47.304520 2025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:47.306078 2009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:28:47.306284 2009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "baeeb821d49c4305856834c1a1c65cc4"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "8d39b7904361fbcc86caf5335bf095f9"
server_key_iv: "cdc46a23e071b1f8a76535ff5174d16f"
server_key_version: "encryptionkey@0"
I20260806 20:28:47.306471 2009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:47.322609 2009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:47.326859 2009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:47.327147 2009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:47.328219 2009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:47.329633 2009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:47.329710 2009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:47.329867 2009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:47.329933 2009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:47.365419 2009 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:44361
I20260806 20:28:47.365473 2151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:44361 every 8 connection(s)
I20260806 20:28:47.366812 2009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:28:47.367885 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 2009
I20260806 20:28:47.368119 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:28:47.368695 17847 external_mini_cluster.cc:1428] Setting key a7139dba694bd1e6ace0df1971dabfd3
Aug 06 20:28:47 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048127, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:47.388572 2154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:47.369883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:41017 (local address 127.17.109.254:46883)
0806 20:28:47.370404 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:47.370411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:47.371537 (+ 1126us) server_negotiation.cc:407] Connection header received
0806 20:28:47.373440 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:47.373445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:47.373541 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:47.373666 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:47.375233 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.376046 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.377627 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.378072 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.378226 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:47.382248 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:47.382253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:47.382257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:47.382293 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:47.385139 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:47.385998 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:47.386004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:47.386008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:47.386169 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:47.386637 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:47.386642 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:47.386645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:47.386931 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:47.387112 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:47.387905 (+ 793us) server_negotiation.cc:299] Negotiation successful
0806 20:28:47.388170 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":268,"threads_started":1}
I20260806 20:28:47.389726 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:47.370504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:41017)
0806 20:28:47.371379 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:47.371419 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:47.373172 (+ 1753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:47.373920 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:47.373929 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:47.374464 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:47.375050 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:47.375067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.376212 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.376217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:47.377461 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:47.377473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.378335 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.378342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:47.378594 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:47.379343 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:47.379367 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:47.381962 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:47.385314 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:47.385324 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:47.385344 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:47.385865 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:47.386284 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:47.386290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:47.386294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:47.386518 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:47.387064 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:47.387074 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:47.387457 (+ 383us) client_negotiation.cc:769] Sending connection context
0806 20:28:47.387818 (+ 361us) client_negotiation.cc:240] Negotiation successful
0806 20:28:47.388448 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":374,"threads_started":1}
I20260806 20:28:47.391618 2152 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46883
I20260806 20:28:47.391952 2152 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:47.392879 2152 heartbeater.cc:511] Master 127.17.109.254:46883 requested a full tablet report, sending...
I20260806 20:28:47.394881 1798 ts_manager.cc:194] Registered new tserver with Master: baeeb821d49c4305856834c1a1c65cc4 (127.17.109.194:44361)
I20260806 20:28:47.396328 1798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:41017
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:28:47.439250 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46883
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:28:47.796784 2159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:28:47.797140 2159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:28:47.797278 2159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:28:47.805801 2159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:28:47.805938 2159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:28:47.806075 2159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:28:47.817937 2159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:28:47.819337 2159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:28:47.820827 2159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:28:47.835834 2168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:47.836305 2165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:28:47.835807 2164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:28:47.842027 2159 server_base.cc:1034] running on GCE node
I20260806 20:28:47.842705 2159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:28:47.843789 2159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:28:47.844995 2159 hybrid_clock.cc:648] HybridClock initialized: now 1786048127844944 us; error 44 us; skew 500 ppm
I20260806 20:28:47.845288 2159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:28:47.847944 2003 heartbeater.cc:503] Master 127.17.109.254:46883 was elected leader, sending a full tablet report...
Aug 06 20:28:47 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048127, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:28:47.848788 2159 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:28:47.850680 2159 webserver.cc:460] Webserver started at http://127.17.109.195:36177/ using document root <none> and password file <none>
I20260806 20:28:47.851459 2159 fs_manager.cc:359] Metadata directory not provided
I20260806 20:28:47.851630 2159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:28:47.851961 2159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:28:47.855451 2159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "cc5d744ba96e41c2ac6d16f9690c369e"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "f2617d83b01f21f24200556e7997ed8b"
server_key_iv: "b43001dc58623956401ea960e4cbe53e"
server_key_version: "encryptionkey@0"
I20260806 20:28:47.856333 2159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "cc5d744ba96e41c2ac6d16f9690c369e"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "f2617d83b01f21f24200556e7997ed8b"
server_key_iv: "b43001dc58623956401ea960e4cbe53e"
server_key_version: "encryptionkey@0"
I20260806 20:28:47.861749 2159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:28:47.865342 2175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:47.866684 2159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:28:47.866868 2159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "cc5d744ba96e41c2ac6d16f9690c369e"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "f2617d83b01f21f24200556e7997ed8b"
server_key_iv: "b43001dc58623956401ea960e4cbe53e"
server_key_version: "encryptionkey@0"
I20260806 20:28:47.867091 2159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:28:47.883037 2159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:28:47.887780 2159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:28:47.888083 2159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:28:47.889139 2159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:28:47.890730 2159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:28:47.890842 2159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:47.890988 2159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:28:47.891062 2159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:28:47.925848 2159 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:39023
I20260806 20:28:47.925904 2293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:39023 every 8 connection(s)
I20260806 20:28:47.927294 2159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:28:47.931757 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 2159
I20260806 20:28:47.931957 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:28:47.932546 17847 external_mini_cluster.cc:1428] Setting key d84b57a99a350bd8682a7f4453bdc7a1
Aug 06 20:28:47 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048127, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:28:47.949187 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:47.930182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:52729 (local address 127.17.109.254:46883)
0806 20:28:47.930693 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:47.930700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:47.931845 (+ 1145us) server_negotiation.cc:407] Connection header received
0806 20:28:47.934212 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:47.934218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:47.934325 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:47.934450 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:47.936108 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.936988 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.938724 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.939170 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.939308 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:47.943009 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:47.943014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:47.943017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:47.943057 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:47.945731 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:47.946475 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:47.946480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:47.946482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:47.946658 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:47.947137 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:47.947141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:47.947143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:47.947539 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:47.947714 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:47.948601 (+ 887us) server_negotiation.cc:299] Negotiation successful
0806 20:28:47.948858 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":225,"threads_started":1}
I20260806 20:28:47.950116 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:47.930784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:52729)
0806 20:28:47.931655 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:47.931719 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:47.933860 (+ 2141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:47.934702 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:47.934710 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:47.935272 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:28:47.935936 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:47.935955 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.937161 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.937166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:47.938554 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:47.938567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.939435 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.939446 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:47.939739 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:47.940458 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:28:47.940481 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:28:47.942795 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:28:47.945888 (+ 3093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:47.945898 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:47.945915 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:47.946346 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:47.946769 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:28:47.946774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:28:47.946777 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:28:47.947013 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:28:47.947788 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:28:47.947795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:28:47.948197 (+ 402us) client_negotiation.cc:769] Sending connection context
0806 20:28:47.948485 (+ 288us) client_negotiation.cc:240] Negotiation successful
0806 20:28:47.948894 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":346,"threads_started":1}
I20260806 20:28:47.951450 2294 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46883
I20260806 20:28:47.951799 2294 heartbeater.cc:464] Registering TS with master...
I20260806 20:28:47.952555 2294 heartbeater.cc:511] Master 127.17.109.254:46883 requested a full tablet report, sending...
I20260806 20:28:47.954010 1798 ts_manager.cc:194] Registered new tserver with Master: cc5d744ba96e41c2ac6d16f9690c369e (127.17.109.195:39023)
I20260806 20:28:47.954831 1798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:52729
I20260806 20:28:47.960834 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:28:47.980448 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:47.966094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60932 (local address 127.17.109.254:46883)
0806 20:28:47.966362 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:47.966369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:47.966692 (+ 323us) server_negotiation.cc:407] Connection header received
0806 20:28:47.966881 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:47.966885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:47.966956 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:47.967046 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:47.969056 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.969866 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.971406 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:47.971973 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:47.972075 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:47.975090 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:47.975094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:47.975097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:47.975132 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:47.977692 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:47.978439 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:47.978443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:47.978445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:47.978577 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:47.978956 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:47.978959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:47.978961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:47.979304 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:47.979487 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:47.979907 (+ 420us) server_negotiation.cc:299] Negotiation successful
0806 20:28:47.980097 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 20:28:47.984866 1798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:28:47.992087 1798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:28:48.017504 2308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.005490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:39023 (local address 127.0.0.1:50766)
0806 20:28:48.007426 (+ 1936us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:48.007474 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:48.007793 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:48.009022 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:48.009033 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:48.009085 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:48.009582 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.009599 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.011788 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.011796 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.014478 (+ 2682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.014492 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.016461 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.016468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.016689 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.016744 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 20:28:48.016890 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 20:28:48.017092 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1682,"thread_start_us":1510,"threads_started":1}
I20260806 20:28:48.018543 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.005807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50766 (local address 127.17.109.195:39023)
0806 20:28:48.006974 (+ 1167us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.006987 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.007622 (+ 635us) server_negotiation.cc:407] Connection header received
0806 20:28:48.008169 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.008177 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.008460 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.008656 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:48.009818 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.011648 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.014648 (+ 3000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.016258 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.016458 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.016660 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.017949 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.018133 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":742,"threads_started":1}
I20260806 20:28:48.019426 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.007994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44361 (local address 127.0.0.1:54774)
0806 20:28:48.011133 (+ 3139us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:48.011151 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:48.011275 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:48.011970 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:48.011973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:48.011997 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:48.012390 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.012401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.014119 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.014124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.016109 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.016122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.018363 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.018389 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.018582 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.018690 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 20:28:48.018787 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 20:28:48.018933 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2998,"mutex_wait_us":57,"thread_start_us":2854,"threads_started":1}
I20260806 20:28:48.019709 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.006360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38512 (local address 127.17.109.193:37011)
0806 20:28:48.008275 (+ 1915us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.008287 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.010541 (+ 2254us) server_negotiation.cc:407] Connection header received
0806 20:28:48.010648 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.010655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.010939 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.011144 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:48.012256 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.013903 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.016778 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.018173 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.018421 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.018592 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.019122 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.019293 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1712,"thread_start_us":1577,"threads_started":1}
I20260806 20:28:48.019455 2309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.005703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37011 (local address 127.0.0.1:38512)
0806 20:28:48.010247 (+ 4544us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:48.010269 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:48.010405 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:48.011151 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:48.011156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:48.011183 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:48.011616 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.011629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.014052 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.014061 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.016587 (+ 2526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.016601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.018421 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.018428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.018689 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.018727 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 20:28:48.018818 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 20:28:48.018972 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4397,"mutex_wait_us":169,"spinlock_wait_cycles":8960,"thread_start_us":4221,"threads_started":1}
I20260806 20:28:48.020273 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.008511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.17.109.194:44361)
0806 20:28:48.010641 (+ 2130us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.010650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.011410 (+ 760us) server_negotiation.cc:407] Connection header received
0806 20:28:48.011497 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.011502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.011770 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.011985 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:48.012597 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.013964 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.016339 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.017865 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.018032 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.018199 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.019702 (+ 1503us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.019913 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1884,"thread_start_us":1777,"threads_started":1}
I20260806 20:28:48.023243 2226 tablet_service.cc:1467] Processing CreateTablet for tablet 800b62b798af41cfb7f943b676b99937 (DEFAULT_TABLE table=test-table [id=81cace5a03d14bb980058121b17ca955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:48.023794 1931 tablet_service.cc:1467] Processing CreateTablet for tablet 800b62b798af41cfb7f943b676b99937 (DEFAULT_TABLE table=test-table [id=81cace5a03d14bb980058121b17ca955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:48.023864 2077 tablet_service.cc:1467] Processing CreateTablet for tablet 800b62b798af41cfb7f943b676b99937 (DEFAULT_TABLE table=test-table [id=81cace5a03d14bb980058121b17ca955]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:28:48.025015 2226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 800b62b798af41cfb7f943b676b99937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:48.025067 1931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 800b62b798af41cfb7f943b676b99937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:48.025219 2077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 800b62b798af41cfb7f943b676b99937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:28:48.037925 2315 tablet_bootstrap.cc:492] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: Bootstrap starting.
I20260806 20:28:48.040570 2316 tablet_bootstrap.cc:492] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Bootstrap starting.
I20260806 20:28:48.041927 2315 tablet_bootstrap.cc:654] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:48.043378 2315 log.cc:824] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:48.044319 2318 tablet_bootstrap.cc:492] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Bootstrap starting.
I20260806 20:28:48.044641 2316 tablet_bootstrap.cc:654] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:48.046228 2316 log.cc:824] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:48.047984 2318 tablet_bootstrap.cc:654] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Neither blocks nor log segments found. Creating new log.
I20260806 20:28:48.049156 2318 log.cc:824] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Log is configured to *not* fsync() on all Append() calls
I20260806 20:28:48.049594 2316 tablet_bootstrap.cc:492] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: No bootstrap required, opened a new log
I20260806 20:28:48.049913 2316 ts_tablet_manager.cc:1397] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 20:28:48.052101 2315 tablet_bootstrap.cc:492] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: No bootstrap required, opened a new log
I20260806 20:28:48.052408 2315 ts_tablet_manager.cc:1397] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260806 20:28:48.054018 2318 tablet_bootstrap.cc:492] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: No bootstrap required, opened a new log
I20260806 20:28:48.054315 2318 ts_tablet_manager.cc:1397] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260806 20:28:48.061179 2316 raft_consensus.cc:348] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.061728 2316 raft_consensus.cc:374] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:48.061868 2316 raft_consensus.cc:729] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5d744ba96e41c2ac6d16f9690c369e, State: Initialized, Role: FOLLOWER
I20260806 20:28:48.062597 2316 consensus_queue.cc:260] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.062952 2318 raft_consensus.cc:348] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.063580 2318 raft_consensus.cc:374] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:48.063238 2315 raft_consensus.cc:348] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.063781 2315 raft_consensus.cc:374] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:28:48.063784 2318 raft_consensus.cc:729] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baeeb821d49c4305856834c1a1c65cc4, State: Initialized, Role: FOLLOWER
I20260806 20:28:48.063954 2315 raft_consensus.cc:729] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dbc7b0bc8d466999992f2942f1d553, State: Initialized, Role: FOLLOWER
I20260806 20:28:48.064121 2294 heartbeater.cc:503] Master 127.17.109.254:46883 was elected leader, sending a full tablet report...
I20260806 20:28:48.064757 2316 ts_tablet_manager.cc:1428] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260806 20:28:48.064559 2318 consensus_queue.cc:260] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.064683 2315 consensus_queue.cc:260] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.066527 2315 ts_tablet_manager.cc:1428] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260806 20:28:48.070523 2152 heartbeater.cc:503] Master 127.17.109.254:46883 was elected leader, sending a full tablet report...
I20260806 20:28:48.073621 2318 ts_tablet_manager.cc:1428] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Time spent starting tablet: real 0.019s user 0.009s sys 0.009s
I20260806 20:28:48.099463 2322 raft_consensus.cc:482] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:28:48.099939 2322 raft_consensus.cc:504] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.102485 2322 leader_election.cc:288] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3dbc7b0bc8d466999992f2942f1d553 (127.17.109.193:37011), baeeb821d49c4305856834c1a1c65cc4 (127.17.109.194:44361)
I20260806 20:28:48.110308 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.103646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:34977 (local address 127.17.109.194:44361)
0806 20:28:48.103923 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.103930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.103952 (+ 22us) server_negotiation.cc:407] Connection header received
0806 20:28:48.104039 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.104045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.104123 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.104227 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:48.104799 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.105817 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.108590 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.109370 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.109556 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.109638 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.109911 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.110041 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 20:28:48.110406 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.103039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44361 (local address 127.17.109.195:34977)
0806 20:28:48.103253 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:48.103276 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:48.103420 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:48.104251 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:48.104255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:48.104309 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:48.104585 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.104599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.106009 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.106015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.108434 (+ 2419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.108451 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.109525 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.109530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.109736 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.109762 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 20:28:48.109868 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 20:28:48.110008 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1,"spinlock_wait_cycles":3328}
I20260806 20:28:48.111290 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.103065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:51097 (local address 127.17.109.193:37011)
0806 20:28:48.103271 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.103276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.104338 (+ 1062us) server_negotiation.cc:407] Connection header received
0806 20:28:48.104566 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.104572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.104662 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.104767 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:28:48.105476 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.106691 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.109161 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.110171 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.110370 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.110452 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.110858 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.111007 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 20:28:48.111462 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.103061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37011 (local address 127.17.109.195:51097)
0806 20:28:48.104205 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0806 20:28:48.104232 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:28:48.104391 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:28:48.104799 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:28:48.104803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:28:48.104829 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:28:48.105234 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.105247 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.106902 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.106908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.109001 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:28:48.109012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.110419 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.110426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:28:48.110647 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.110713 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 20:28:48.110829 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 20:28:48.110988 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"mutex_wait_us":225,"spinlock_wait_cycles":11776,"thread_start_us":514,"threads_started":1}
I20260806 20:28:48.111677 2099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "800b62b798af41cfb7f943b676b99937" candidate_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baeeb821d49c4305856834c1a1c65cc4" is_pre_election: true
I20260806 20:28:48.112272 2099 raft_consensus.cc:2454] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc5d744ba96e41c2ac6d16f9690c369e in term 0.
I20260806 20:28:48.112771 1951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "800b62b798af41cfb7f943b676b99937" candidate_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dbc7b0bc8d466999992f2942f1d553" is_pre_election: true
I20260806 20:28:48.113201 2177 leader_election.cc:302] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: baeeb821d49c4305856834c1a1c65cc4, cc5d744ba96e41c2ac6d16f9690c369e; no voters:
I20260806 20:28:48.113317 1951 raft_consensus.cc:2454] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc5d744ba96e41c2ac6d16f9690c369e in term 0.
I20260806 20:28:48.113670 2322 raft_consensus.cc:2781] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:28:48.113868 2322 raft_consensus.cc:482] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:28:48.113996 2322 raft_consensus.cc:3037] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:48.116480 2322 raft_consensus.cc:504] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.117086 2322 leader_election.cc:288] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [CANDIDATE]: Term 1 election: Requested vote from peers b3dbc7b0bc8d466999992f2942f1d553 (127.17.109.193:37011), baeeb821d49c4305856834c1a1c65cc4 (127.17.109.194:44361)
I20260806 20:28:48.117659 1951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "800b62b798af41cfb7f943b676b99937" candidate_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dbc7b0bc8d466999992f2942f1d553"
I20260806 20:28:48.117659 2099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "800b62b798af41cfb7f943b676b99937" candidate_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baeeb821d49c4305856834c1a1c65cc4"
I20260806 20:28:48.117892 2099 raft_consensus.cc:3037] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:28:48.117892 1951 raft_consensus.cc:3037] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Advancing to term 1
W20260806 20:28:48.118898 2153 tablet.cc:2369] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:28:48.121399 1951 raft_consensus.cc:2454] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc5d744ba96e41c2ac6d16f9690c369e in term 1.
I20260806 20:28:48.121399 2099 raft_consensus.cc:2454] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc5d744ba96e41c2ac6d16f9690c369e in term 1.
I20260806 20:28:48.121980 2176 leader_election.cc:302] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b3dbc7b0bc8d466999992f2942f1d553, baeeb821d49c4305856834c1a1c65cc4, cc5d744ba96e41c2ac6d16f9690c369e; no voters:
I20260806 20:28:48.122396 2322 raft_consensus.cc:2781] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:28:48.122855 2322 raft_consensus.cc:686] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 1 LEADER]: Becoming Leader. State: Replica: cc5d744ba96e41c2ac6d16f9690c369e, State: Running, Role: LEADER
I20260806 20:28:48.123365 2322 consensus_queue.cc:237] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:28:48.128433 1795 catalog_manager.cc:5495] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc5d744ba96e41c2ac6d16f9690c369e (127.17.109.195). New cstate: current_term: 1 leader_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260806 20:28:48.176664 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.169104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.17.109.195:39023)
0806 20:28:48.169318 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.169324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.169442 (+ 118us) server_negotiation.cc:407] Connection header received
0806 20:28:48.169688 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.169692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.169780 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.169960 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:28:48.170348 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.171139 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.172738 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.173363 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.173531 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.174388 (+ 857us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:28:48.175233 (+ 845us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:28:48.175405 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.176195 (+ 790us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.176306 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 20:28:48.180330 2295 tablet.cc:2369] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:28:48.190136 2099 raft_consensus.cc:1261] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 1 FOLLOWER]: Refusing update from remote peer cc5d744ba96e41c2ac6d16f9690c369e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:48.190260 1951 raft_consensus.cc:1261] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 1 FOLLOWER]: Refusing update from remote peer cc5d744ba96e41c2ac6d16f9690c369e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:28:48.191215 2327 consensus_queue.cc:1035] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:48.191586 2322 consensus_queue.cc:1035] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [LEADER]: Connected to new peer: Peer: permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:28:48.207543 2330 mvcc.cc:205] Tried to move back new op lower bound from 7315653133055094784 to 7315653132797313024. Current Snapshot: MvccSnapshot[applied={T|T < 7315653133055094784}]
I20260806 20:28:48.222131 1798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:28:48.222415 1798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:28:48.226227 1798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Sending DeleteTablet for 3 replicas of tablet 800b62b798af41cfb7f943b676b99937
I20260806 20:28:48.227618 1931 tablet_service.cc:1514] Processing DeleteTablet for tablet 800b62b798af41cfb7f943b676b99937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:48 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:38512
I20260806 20:28:48.227838 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 1865
I20260806 20:28:48.227932 2226 tablet_service.cc:1514] Processing DeleteTablet for tablet 800b62b798af41cfb7f943b676b99937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:48 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:50766
I20260806 20:28:48.228673 2077 tablet_service.cc:1514] Processing DeleteTablet for tablet 800b62b798af41cfb7f943b676b99937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:28:48 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:54774
I20260806 20:28:48.238122 2347 tablet_replica.cc:330] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: stopping tablet replica
I20260806 20:28:48.239696 2347 raft_consensus.cc:2229] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:28:48.242419 2347 raft_consensus.cc:2258] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:48.245191 2347 ts_tablet_manager.cc:1905] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:48.246034 2344 tablet_replica.cc:330] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: stopping tablet replica
I20260806 20:28:48.246743 2344 raft_consensus.cc:2229] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:28:48.248225 2345 tablet_replica.cc:330] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: stopping tablet replica
I20260806 20:28:48.248971 2344 raft_consensus.cc:2258] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:48.248916 2345 raft_consensus.cc:2229] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:28:48.249766 2345 raft_consensus.cc:2258] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:28:48.252259 2344 ts_tablet_manager.cc:1905] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:48.252893 2345 ts_tablet_manager.cc:1905] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:28:48.255932 2347 ts_tablet_manager.cc:1918] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:48.256188 2347 log.cc:1196] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/800b62b798af41cfb7f943b676b99937
I20260806 20:28:48.256860 2347 ts_tablet_manager.cc:1939] T 800b62b798af41cfb7f943b676b99937 P baeeb821d49c4305856834c1a1c65cc4: Deleting consensus metadata
I20260806 20:28:48.259073 1782 catalog_manager.cc:4841] TS baeeb821d49c4305856834c1a1c65cc4 (127.17.109.194:44361): tablet 800b62b798af41cfb7f943b676b99937 (table test-table [id=81cace5a03d14bb980058121b17ca955]) successfully deleted
I20260806 20:28:48.259475 2344 ts_tablet_manager.cc:1918] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:48.259755 2344 log.cc:1196] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/800b62b798af41cfb7f943b676b99937
I20260806 20:28:48.260365 2344 ts_tablet_manager.cc:1939] T 800b62b798af41cfb7f943b676b99937 P b3dbc7b0bc8d466999992f2942f1d553: Deleting consensus metadata
I20260806 20:28:48.262459 1781 catalog_manager.cc:4841] TS b3dbc7b0bc8d466999992f2942f1d553 (127.17.109.193:37011): tablet 800b62b798af41cfb7f943b676b99937 (table test-table [id=81cace5a03d14bb980058121b17ca955]) successfully deleted
Aug 06 20:28:48 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:28:48.263863 2345 ts_tablet_manager.cc:1918] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:28:48.264081 2345 log.cc:1196] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/800b62b798af41cfb7f943b676b99937
I20260806 20:28:48.264662 2345 ts_tablet_manager.cc:1939] T 800b62b798af41cfb7f943b676b99937 P cc5d744ba96e41c2ac6d16f9690c369e: Deleting consensus metadata
I20260806 20:28:48.266573 1782 catalog_manager.cc:4841] TS cc5d744ba96e41c2ac6d16f9690c369e (127.17.109.195:39023): tablet 800b62b798af41cfb7f943b676b99937 (table test-table [id=81cace5a03d14bb980058121b17ca955]) successfully deleted
I20260806 20:28:48.271656 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.229452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.17.109.193:37011)
0806 20:28:48.229640 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.229646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.241477 (+ 11831us) server_negotiation.cc:407] Connection header received
0806 20:28:48.243483 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.243492 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.243662 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.243759 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:48.249540 (+ 5781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.250737 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.252869 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.253545 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.253652 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.264606 (+ 10954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:48.264620 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:48.264634 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:48.264680 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:48.268417 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:48.269317 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:48.269326 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:48.269328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:48.269509 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:48.269918 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:48.269922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:48.269926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:48.270347 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:48.270666 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.271023 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.271251 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 20:28:48.272450 1997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:48.364753 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 1865
I20260806 20:28:48.392346 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 2009
Aug 06 20:28:48 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:28:48.411914 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.393274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.17.109.194:44361)
0806 20:28:48.393464 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.393471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.393525 (+ 54us) server_negotiation.cc:407] Connection header received
0806 20:28:48.393711 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.393718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.393851 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.393979 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:48.395860 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.397101 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.399098 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.399815 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.399939 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.405661 (+ 5722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:48.405672 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:48.405686 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:48.405724 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:48.408802 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:48.409671 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:48.409679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:48.409681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:48.409803 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:48.410204 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:48.410207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:48.410209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:48.410639 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:48.410855 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.411252 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.411442 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:28:48.412588 2146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:48.508476 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 2009
I20260806 20:28:48.529255 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 2159
Aug 06 20:28:48 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:28:48.550307 2310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:28:48.530166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40140 (local address 127.17.109.195:39023)
0806 20:28:48.530365 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 20:28:48.530370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:28:48.530475 (+ 105us) server_negotiation.cc:407] Connection header received
0806 20:28:48.530727 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:28:48.530731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:28:48.530814 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:28:48.530919 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:28:48.532834 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.533981 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.536586 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:28:48.537066 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:28:48.537173 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:28:48.543328 (+ 6155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:28:48.543343 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:28:48.543360 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:28:48.543421 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:28:48.546713 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:48.548071 (+ 1358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:48.548078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:48.548080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:48.548253 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:28:48.548761 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:28:48.548764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:28:48.548766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:28:48.549138 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:28:48.549384 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 20:28:48.549761 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 20:28:48.549958 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 20:28:48.551056 2289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:48.678669 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 2159
I20260806 20:28:48.702123 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 1765
I20260806 20:28:48.703249 1831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:28:48.816625 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 1765
I20260806 20:29:08.842217 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46883
--webserver_interface=127.17.109.254
--webserver_port=42853
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:09.199041 2353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:09.199357 2353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:09.199534 2353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:09.207055 2353 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 20:29:09.207162 2353 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 20:29:09.207209 2353 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 20:29:09.207286 2353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:29:09.207366 2353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:09.207482 2353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:09.207590 2353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:29:09.207702 2353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:29:09.218062 2353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=42853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:09.219456 2353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:09.220873 2353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:09.247153 2362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:09.247262 2360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:09.247169 2364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:09.248180 2353 server_base.cc:1034] running on GCE node
I20260806 20:29:09.249043 2353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:09.250617 2353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:09.251866 2353 hybrid_clock.cc:648] HybridClock initialized: now 1786048149251789 us; error 27 us; skew 500 ppm
I20260806 20:29:09.252363 2353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:09 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:09.256316 2353 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:29:09.257987 2353 webserver.cc:460] Webserver started at http://127.17.109.254:42853/ using document root <none> and password file <none>
I20260806 20:29:09.258792 2353 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:09.258905 2353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:09.265232 2353 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 20:29:09.268349 2371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:09.269820 2353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:09.270022 2353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "a1f104883451419d8828e780a72e58bc"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "5bc853282a857b4269b0d8d62b47a62f"
server_key_iv: "263bb76a10a91476f179bb301f672efa"
server_key_version: "encryptionkey@0"
I20260806 20:29:09.271130 2353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:09.294799 2353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:09.300009 2353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:09.300331 2353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:09.321926 2353 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:46883
I20260806 20:29:09.321987 2427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:46883 every 8 connection(s)
I20260806 20:29:09.323657 2353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:29:09.324788 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 2353
I20260806 20:29:09.325826 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:37011
--local_ip_for_outbound_sockets=127.17.109.193
--tserver_master_addrs=127.17.109.254:46883
--webserver_port=39391
--webserver_interface=127.17.109.193
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:29:09.331451 2429 sys_catalog.cc:263] Verifying existing consensus state
I20260806 20:29:09.335031 2429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Bootstrap starting.
I20260806 20:29:09.355990 2429 log.cc:824] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:09.369307 2429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:29:09.369905 2429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Bootstrap complete.
I20260806 20:29:09.379846 2429 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:29:09.380759 2429 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f104883451419d8828e780a72e58bc, State: Initialized, Role: FOLLOWER
I20260806 20:29:09.381453 2429 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:29:09.381723 2429 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:29:09.381865 2429 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:29:09.382050 2429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 1 FOLLOWER]: Advancing to term 2
I20260806 20:29:09.386066 2429 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:29:09.386619 2429 leader_election.cc:302] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1f104883451419d8828e780a72e58bc; no voters:
I20260806 20:29:09.387326 2429 leader_election.cc:288] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 20:29:09.387468 2433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 2 FOLLOWER]: Leader election won for term 2
I20260806 20:29:09.388805 2433 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 2 LEADER]: Becoming Leader. State: Replica: a1f104883451419d8828e780a72e58bc, State: Running, Role: LEADER
I20260806 20:29:09.389462 2433 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:29:09.389945 2429 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:29:09.392093 2434 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a1f104883451419d8828e780a72e58bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } } }
I20260806 20:29:09.392508 2434 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:09.393065 2435 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1f104883451419d8828e780a72e58bc. Latest consensus state: current_term: 2 leader_uuid: "a1f104883451419d8828e780a72e58bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } } }
I20260806 20:29:09.393397 2435 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:09.404945 2439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:29:09.414633 2439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:29:09.416555 2439 catalog_manager.cc:1259] Loaded cluster ID: 0e7bc51c390243099d1947ecb3c0dfa6
I20260806 20:29:09.416766 2439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:29:09.421319 2452 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 20:29:09.422262 2439 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:29:09.423420 2439 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Loaded TSK: 0
I20260806 20:29:09.448923 2439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Generated new TSK 1
I20260806 20:29:09.449313 2439 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 20:29:09.719338 2431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:09.719728 2431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:09.719910 2431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:09.727210 2431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:09.727391 2431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:09.727613 2431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:29:09.737529 2431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:37011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=39391
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:09.738920 2431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:09.740427 2431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:09.755456 2458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:09.755429 2459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:09.755875 2461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:09.756814 2431 server_base.cc:1034] running on GCE node
I20260806 20:29:09.757448 2431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:09.758545 2431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:09.759744 2431 hybrid_clock.cc:648] HybridClock initialized: now 1786048149759694 us; error 43 us; skew 500 ppm
I20260806 20:29:09.760080 2431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:09 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:09.763465 2431 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:29:09.765244 2431 webserver.cc:460] Webserver started at http://127.17.109.193:39391/ using document root <none> and password file <none>
I20260806 20:29:09.765960 2431 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:09.766110 2431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:09.771924 2431 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260806 20:29:09.774693 2470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:09.776317 2431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:09.776489 2431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "b3dbc7b0bc8d466999992f2942f1d553"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "f57e35a9eb53e5747f08b8bfc6cc7f3c"
server_key_iv: "0cf1a719207111e08d2476ec23bca448"
server_key_version: "encryptionkey@0"
I20260806 20:29:09.777470 2431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:09.795382 2431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:09.799875 2431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:09.800153 2431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:09.801142 2431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:09.802587 2431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:09.802711 2431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:09.802846 2431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:09.802934 2431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:09.837522 2431 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:37011
I20260806 20:29:09.837615 2598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:37011 every 8 connection(s)
I20260806 20:29:09.838996 2431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:29:09.848112 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 2431
I20260806 20:29:09.848790 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:44361
--local_ip_for_outbound_sockets=127.17.109.194
--tserver_master_addrs=127.17.109.254:46883
--webserver_port=34881
--webserver_interface=127.17.109.194
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:29:09 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048149, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:09.864820 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:09.841966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:44015 (local address 127.17.109.254:46883)
0806 20:29:09.842676 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:09.842695 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:09.843182 (+ 487us) server_negotiation.cc:407] Connection header received
0806 20:29:09.845384 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:09.845397 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:09.845726 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:09.846085 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:09.847750 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:09.849178 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:09.851763 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:09.852453 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:09.852603 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:09.856987 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:09.856994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:09.857008 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:09.857052 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:09.860075 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:09.860847 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:09.860854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:09.860860 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:09.861019 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:09.861490 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:09.861494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:09.861496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:09.861981 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:09.862185 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:09.863042 (+ 857us) server_negotiation.cc:299] Negotiation successful
0806 20:29:09.863435 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":281,"threads_started":1}
I20260806 20:29:09.864802 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:09.842138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:44015)
0806 20:29:09.843001 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:09.843046 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:09.844836 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:09.846241 (+ 1405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:09.846252 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:09.846884 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:09.847423 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:09.847440 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:09.849357 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:09.849366 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:09.851586 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:09.851606 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:09.852710 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:09.852719 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:09.853070 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:09.853884 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:29:09.853919 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:09.856694 (+ 2775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:09.860221 (+ 3527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:09.860232 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:09.860247 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:09.860674 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:09.861150 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:09.861155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:09.861158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:09.861358 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:09.862128 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:09.862136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:09.862488 (+ 352us) client_negotiation.cc:769] Sending connection context
0806 20:29:09.862801 (+ 313us) client_negotiation.cc:240] Negotiation successful
0806 20:29:09.863416 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":354,"threads_started":1}
I20260806 20:29:09.867813 2599 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46883
I20260806 20:29:09.868175 2599 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:09.869138 2599 heartbeater.cc:511] Master 127.17.109.254:46883 requested a full tablet report, sending...
I20260806 20:29:09.871510 2388 ts_manager.cc:194] Registered new tserver with Master: b3dbc7b0bc8d466999992f2942f1d553 (127.17.109.193:37011)
I20260806 20:29:09.873024 2388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:44015
W20260806 20:29:10.199429 2603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:10.199790 2603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:10.199975 2603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:10.207446 2603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:10.207547 2603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:10.207739 2603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:29:10.217504 2603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:44361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=34881
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:10.218856 2603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:10.220309 2603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:10.234028 2610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:10.234108 2612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:10.234153 2609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:10.235198 2603 server_base.cc:1034] running on GCE node
I20260806 20:29:10.235869 2603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:10.236799 2603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:10.238008 2603 hybrid_clock.cc:648] HybridClock initialized: now 1786048150237960 us; error 37 us; skew 500 ppm
I20260806 20:29:10.238287 2603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:10 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:10.241586 2603 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:29:10.243263 2603 webserver.cc:460] Webserver started at http://127.17.109.194:34881/ using document root <none> and password file <none>
I20260806 20:29:10.244017 2603 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:10.244170 2603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:10.249598 2603 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 20:29:10.252415 2619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:10.253765 2603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:10.253959 2603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "baeeb821d49c4305856834c1a1c65cc4"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "8d39b7904361fbcc86caf5335bf095f9"
server_key_iv: "cdc46a23e071b1f8a76535ff5174d16f"
server_key_version: "encryptionkey@0"
I20260806 20:29:10.255033 2603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:10.268760 2603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:10.272787 2603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:10.273087 2603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:10.274145 2603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:10.275628 2603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:10.275805 2603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260806 20:29:10.275954 2603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:10.276057 2603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:10.310766 2603 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:44361
I20260806 20:29:10.310856 2742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:44361 every 8 connection(s)
I20260806 20:29:10.312268 2603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:29:10.321861 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 2603
I20260806 20:29:10.322664 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:39023
--local_ip_for_outbound_sockets=127.17.109.195
--tserver_master_addrs=127.17.109.254:46883
--webserver_port=36177
--webserver_interface=127.17.109.195
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 20:29:10 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:10.332708 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.314811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:46003 (local address 127.17.109.254:46883)
0806 20:29:10.314993 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:10.314997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:10.316161 (+ 1164us) server_negotiation.cc:407] Connection header received
0806 20:29:10.318047 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:10.318053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:10.318144 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:10.318256 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:10.319860 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.320561 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.322282 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.322712 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.322873 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.326531 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:10.326537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:10.326540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:10.326573 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:10.329420 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:10.330198 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:10.330203 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:10.330205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:10.330334 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:10.330795 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:10.330805 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:10.330807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:10.331111 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:10.331274 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:10.332081 (+ 807us) server_negotiation.cc:299] Negotiation successful
0806 20:29:10.332365 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 20:29:10.333936 2745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.315273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:46003)
0806 20:29:10.316001 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:10.316045 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:10.317771 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:10.318518 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:10.318526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:10.319039 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:10.319694 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.319714 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.320719 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.320725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.322130 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.322142 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.323153 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.323158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.323356 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.324099 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:29:10.324120 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:10.326290 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:10.329575 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:10.329583 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:10.329599 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:10.330056 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:10.330453 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:10.330459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:10.330463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:10.330665 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:10.331246 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:10.331254 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:10.331694 (+ 440us) client_negotiation.cc:769] Sending connection context
0806 20:29:10.331980 (+ 286us) client_negotiation.cc:240] Negotiation successful
0806 20:29:10.332563 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":319,"threads_started":1}
I20260806 20:29:10.335522 2743 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46883
I20260806 20:29:10.335903 2743 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:10.336836 2743 heartbeater.cc:511] Master 127.17.109.254:46883 requested a full tablet report, sending...
I20260806 20:29:10.338675 2388 ts_manager.cc:194] Registered new tserver with Master: baeeb821d49c4305856834c1a1c65cc4 (127.17.109.194:44361)
I20260806 20:29:10.339977 2388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:46003
W20260806 20:29:10.673173 2746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:10.673516 2746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:10.673699 2746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:10.681953 2746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:10.682111 2746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:10.682312 2746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:29:10.692418 2746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:39023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=36177
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:10.693816 2746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:10.695215 2746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:10.709072 2752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:10.709133 2751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:10.709322 2754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:10.710283 2746 server_base.cc:1034] running on GCE node
I20260806 20:29:10.710883 2746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:10.711894 2746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:10.713117 2746 hybrid_clock.cc:648] HybridClock initialized: now 1786048150713069 us; error 40 us; skew 500 ppm
I20260806 20:29:10.713491 2746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:10 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:10.723654 2746 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:29:10.725286 2746 webserver.cc:460] Webserver started at http://127.17.109.195:36177/ using document root <none> and password file <none>
I20260806 20:29:10.726079 2746 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:10.726249 2746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:10.731859 2746 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 20:29:10.734671 2761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:10.736135 2746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:10.736336 2746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "cc5d744ba96e41c2ac6d16f9690c369e"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "f2617d83b01f21f24200556e7997ed8b"
server_key_iv: "b43001dc58623956401ea960e4cbe53e"
server_key_version: "encryptionkey@0"
I20260806 20:29:10.737314 2746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:10.766764 2746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:10.771318 2746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:10.771657 2746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:10.772655 2746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:10.773991 2746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:10.774130 2746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:10.774256 2746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:10.774346 2746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:10.809623 2746 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:39023
I20260806 20:29:10.809719 2883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:39023 every 8 connection(s)
I20260806 20:29:10.811084 2746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:29:10.818657 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 2746
Aug 06 20:29:10 dist-test-slave-6k22 krb5kdc[1751](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048150, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:10.832798 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.813791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:54409 (local address 127.17.109.254:46883)
0806 20:29:10.814009 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:10.814013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:10.815182 (+ 1169us) server_negotiation.cc:407] Connection header received
0806 20:29:10.816991 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:10.816995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:10.817062 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:10.817150 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:10.818655 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.819732 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.821628 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.822388 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.822537 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.826256 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:10.826262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:10.826265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:10.826326 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:10.829380 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:10.830299 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:10.830310 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:10.830314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:10.830480 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:10.830918 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:10.830922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:10.830924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:10.831333 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:10.831498 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:10.832258 (+ 760us) server_negotiation.cc:299] Negotiation successful
0806 20:29:10.832498 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"spinlock_wait_cycles":5888}
I20260806 20:29:10.834066 2887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.814307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:54409)
0806 20:29:10.815025 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:10.815064 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:10.816732 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:10.817381 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:10.817389 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:10.817890 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:10.818468 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.818485 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.819986 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.819996 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.821469 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.821495 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.822614 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.822619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.822833 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.823606 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:29:10.823648 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:10.826026 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:10.829534 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:10.829541 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:10.829556 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:10.830176 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:10.830594 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:10.830597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:10.830600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:10.830788 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:10.831474 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:10.831480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:10.831885 (+ 405us) client_negotiation.cc:769] Sending connection context
0806 20:29:10.832129 (+ 244us) client_negotiation.cc:240] Negotiation successful
0806 20:29:10.832644 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":282,"threads_started":1}
I20260806 20:29:10.835840 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.819712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47050 (local address 127.17.109.254:46883)
0806 20:29:10.820296 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:10.820303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:10.820619 (+ 316us) server_negotiation.cc:407] Connection header received
0806 20:29:10.820858 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:10.820864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:10.820934 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:10.821059 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:10.823156 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.823968 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.825680 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.826101 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.826244 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.829184 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:10.829191 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:10.829196 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:10.829241 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:10.832402 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:10.833250 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:10.833253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:10.833256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:10.833381 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:10.833811 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:10.833815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:10.833817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:10.834064 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:10.834339 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:10.835174 (+ 835us) server_negotiation.cc:299] Negotiation successful
0806 20:29:10.835417 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"spinlock_wait_cycles":5376,"thread_start_us":315,"threads_started":1}
I20260806 20:29:10.835850 2885 heartbeater.cc:347] Connected to a master server at 127.17.109.254:46883
I20260806 20:29:10.836185 2885 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:10.837098 2885 heartbeater.cc:511] Master 127.17.109.254:46883 requested a full tablet report, sending...
I20260806 20:29:10.838897 2388 ts_manager.cc:194] Registered new tserver with Master: cc5d744ba96e41c2ac6d16f9690c369e (127.17.109.195:39023)
I20260806 20:29:10.839828 2388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:54409
I20260806 20:29:10.851025 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:29:10.857869 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.852122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.17.109.254:46883)
0806 20:29:10.852423 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:10.852432 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:10.852754 (+ 322us) server_negotiation.cc:407] Connection header received
0806 20:29:10.852937 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:10.852941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:10.853076 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:10.853207 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:29:10.853614 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.854351 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.856026 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.856458 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.856552 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.856986 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:29:10.857276 (+ 290us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 20:29:10.857582 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47066: token expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 20:29:10.858019 2888 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47066: token expired
W20260806 20:29:10.858659 2301 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.109.254:46883)
I20260806 20:29:10.872450 2888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.859096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.17.109.254:46883)
0806 20:29:10.859288 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:10.859292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:10.859309 (+ 17us) server_negotiation.cc:407] Connection header received
0806 20:29:10.859489 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:10.859493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:10.859611 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:10.859693 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:10.861020 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.861747 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.863250 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.863768 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.863865 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.866635 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:10.866642 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:10.866649 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:10.866711 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:10.869574 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:10.870354 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:10.870361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:10.870365 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:10.870505 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:10.870915 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:10.870920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:10.870924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:10.871240 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:10.871496 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:10.871875 (+ 379us) server_negotiation.cc:299] Negotiation successful
0806 20:29:10.872104 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 20:29:10.877345 2599 heartbeater.cc:503] Master 127.17.109.254:46883 was elected leader, sending a full tablet report...
I20260806 20:29:10.876116 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:29:10.884256 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:29:10.911476 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.901217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.17.109.194:44361)
0806 20:29:10.901811 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:10.901819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:10.903372 (+ 1553us) server_negotiation.cc:407] Connection header received
0806 20:29:10.903711 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:10.903719 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:10.903937 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:10.904100 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:10.904987 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.906418 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.908589 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.910081 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.910256 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.910419 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:10.910985 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 20:29:10.911178 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":326,"threads_started":1}
I20260806 20:29:10.911525 2896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.896293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37011 (local address 127.0.0.1:54268)
0806 20:29:10.900187 (+ 3894us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:10.900232 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:10.900413 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:10.902232 (+ 1819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:10.902242 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:10.902288 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:10.903724 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.903741 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.905547 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.905555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.908588 (+ 3033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.908598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.910368 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.910375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.910595 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.910743 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 20:29:10.910866 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 20:29:10.911091 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3634,"mutex_wait_us":156,"thread_start_us":4448,"threads_started":1}
I20260806 20:29:10.912112 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.897140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.17.109.195:39023)
0806 20:29:10.898107 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:10.898120 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:10.901159 (+ 3039us) server_negotiation.cc:407] Connection header received
0806 20:29:10.901511 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:10.901523 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:10.901788 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:10.901962 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:10.904198 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.905734 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.908718 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.910082 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.910283 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.910430 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:10.911272 (+ 842us) server_negotiation.cc:299] Negotiation successful
0806 20:29:10.911450 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":515,"threads_started":1}
I20260806 20:29:10.912314 2901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.901855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44361 (local address 127.0.0.1:36694)
0806 20:29:10.903248 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:10.903269 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:10.903478 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:10.904372 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:10.904377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:10.904400 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:10.904808 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.904819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.906565 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.906570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.908415 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.908430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.910481 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.910487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.910683 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.910802 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 20:29:10.911810 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0806 20:29:10.911951 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1211,"mutex_wait_us":73,"thread_start_us":311,"threads_started":1}
I20260806 20:29:10.912341 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.896597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.17.109.193:37011)
0806 20:29:10.901549 (+ 4952us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:10.901559 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:10.901594 (+ 35us) server_negotiation.cc:407] Connection header received
0806 20:29:10.901693 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:10.901701 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:10.901927 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:10.902398 (+ 471us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:10.903923 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.905416 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.908763 (+ 3347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.909712 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.909962 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.910179 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:10.911821 (+ 1642us) server_negotiation.cc:299] Negotiation successful
0806 20:29:10.912007 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4766,"thread_start_us":4634,"threads_started":1}
I20260806 20:29:10.912909 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:10.896400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:39023 (local address 127.0.0.1:41662)
0806 20:29:10.901041 (+ 4641us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:10.901061 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:10.901262 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:10.902336 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:10.902343 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:10.902378 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:10.903988 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.904007 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.905899 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.905904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.908544 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:10.908564 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:10.910368 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:10.910375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:10.910893 (+ 518us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:10.910920 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 20:29:10.912396 (+ 1476us) client_negotiation.cc:240] Negotiation successful
0806 20:29:10.912550 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4484,"mutex_wait_us":446,"thread_start_us":3699,"threads_started":1}
I20260806 20:29:10.916495 2814 tablet_service.cc:1467] Processing CreateTablet for tablet edb26bc76778489b9042b8ee1761b16a (DEFAULT_TABLE table=test-table [id=5c1136697b994720a2a3c4af1ceaa774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:10.916495 2675 tablet_service.cc:1467] Processing CreateTablet for tablet edb26bc76778489b9042b8ee1761b16a (DEFAULT_TABLE table=test-table [id=5c1136697b994720a2a3c4af1ceaa774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:10.917627 2527 tablet_service.cc:1467] Processing CreateTablet for tablet edb26bc76778489b9042b8ee1761b16a (DEFAULT_TABLE table=test-table [id=5c1136697b994720a2a3c4af1ceaa774]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:10.918036 2814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb26bc76778489b9042b8ee1761b16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:10.918030 2675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb26bc76778489b9042b8ee1761b16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:10.918804 2527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet edb26bc76778489b9042b8ee1761b16a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:10.932188 2902 tablet_bootstrap.cc:492] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Bootstrap starting.
I20260806 20:29:10.935389 2902 tablet_bootstrap.cc:654] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:10.936471 2902 log.cc:824] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:10.938120 2903 tablet_bootstrap.cc:492] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Bootstrap starting.
I20260806 20:29:10.939785 2904 tablet_bootstrap.cc:492] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Bootstrap starting.
I20260806 20:29:10.940768 2902 tablet_bootstrap.cc:492] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: No bootstrap required, opened a new log
I20260806 20:29:10.941097 2902 ts_tablet_manager.cc:1397] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260806 20:29:10.942814 2904 tablet_bootstrap.cc:654] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:10.944008 2904 log.cc:824] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:10.947006 2903 tablet_bootstrap.cc:654] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:10.947319 2904 tablet_bootstrap.cc:492] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: No bootstrap required, opened a new log
I20260806 20:29:10.947633 2904 ts_tablet_manager.cc:1397] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260806 20:29:10.948388 2903 log.cc:824] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:10.949929 2902 raft_consensus.cc:348] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:10.950369 2902 raft_consensus.cc:374] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:10.950476 2902 raft_consensus.cc:729] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dbc7b0bc8d466999992f2942f1d553, State: Initialized, Role: FOLLOWER
I20260806 20:29:10.951086 2902 consensus_queue.cc:260] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:10.951596 2903 tablet_bootstrap.cc:492] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: No bootstrap required, opened a new log
I20260806 20:29:10.951937 2903 ts_tablet_manager.cc:1397] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260806 20:29:10.953962 2902 ts_tablet_manager.cc:1428] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 20:29:10.958094 2904 raft_consensus.cc:348] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:10.958578 2904 raft_consensus.cc:374] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:10.958691 2904 raft_consensus.cc:729] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5d744ba96e41c2ac6d16f9690c369e, State: Initialized, Role: FOLLOWER
I20260806 20:29:10.959322 2904 consensus_queue.cc:260] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:10.960448 2885 heartbeater.cc:503] Master 127.17.109.254:46883 was elected leader, sending a full tablet report...
I20260806 20:29:10.961145 2904 ts_tablet_manager.cc:1428] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 20:29:10.962285 2903 raft_consensus.cc:348] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:10.962673 2903 raft_consensus.cc:374] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:10.962776 2903 raft_consensus.cc:729] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baeeb821d49c4305856834c1a1c65cc4, State: Initialized, Role: FOLLOWER
I20260806 20:29:10.963416 2903 consensus_queue.cc:260] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:10.964694 2743 heartbeater.cc:503] Master 127.17.109.254:46883 was elected leader, sending a full tablet report...
I20260806 20:29:10.965235 2903 ts_tablet_manager.cc:1428] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260806 20:29:11.062811 2886 tablet.cc:2369] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:29:11.064188 2744 tablet.cc:2369] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:29:11.077714 2910 raft_consensus.cc:482] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:29:11.078066 2910 raft_consensus.cc:504] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:11.079902 2910 leader_election.cc:288] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3dbc7b0bc8d466999992f2942f1d553 (127.17.109.193:37011), baeeb821d49c4305856834c1a1c65cc4 (127.17.109.194:44361)
I20260806 20:29:11.087731 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:11.080465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:55455 (local address 127.17.109.193:37011)
0806 20:29:11.080723 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:11.080730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:11.080998 (+ 268us) server_negotiation.cc:407] Connection header received
0806 20:29:11.081231 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:11.081237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:11.081327 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:11.081427 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:11.081885 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.082995 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.085481 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.086711 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.086925 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:11.087003 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:11.087318 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 20:29:11.087421 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 20:29:11.087884 2887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:11.080474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37011 (local address 127.17.109.195:55455)
0806 20:29:11.080856 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:11.080883 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:11.081053 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:11.081489 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:11.081495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:11.081543 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:11.081743 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:11.081751 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.083150 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.083154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:11.085301 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:11.085317 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.086889 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.086893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:11.087129 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:11.087170 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 20:29:11.087276 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 20:29:11.087464 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":1,"spinlock_wait_cycles":21376}
I20260806 20:29:11.087925 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:11.080474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:32877 (local address 127.17.109.194:44361)
0806 20:29:11.080752 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:11.080761 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:11.081505 (+ 744us) server_negotiation.cc:407] Connection header received
0806 20:29:11.081677 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:11.081683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:11.081755 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:11.081857 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:11.082372 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.083340 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.085662 (+ 2322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.086958 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.087115 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:11.087162 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:11.087501 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 20:29:11.087658 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 20:29:11.088007 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:11.080537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44361 (local address 127.17.109.195:32877)
0806 20:29:11.081372 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:11.081385 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:11.081537 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:11.081880 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:11.081885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:11.081914 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:11.082217 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:11.082224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.083460 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.083464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:11.085520 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:11.085530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.087166 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.087172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:11.087349 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:11.087378 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 20:29:11.087458 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 20:29:11.087590 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":439,"threads_started":1}
I20260806 20:29:11.089174 2695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb26bc76778489b9042b8ee1761b16a" candidate_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baeeb821d49c4305856834c1a1c65cc4" is_pre_election: true
I20260806 20:29:11.089174 2547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb26bc76778489b9042b8ee1761b16a" candidate_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dbc7b0bc8d466999992f2942f1d553" is_pre_election: true
I20260806 20:29:11.089679 2695 raft_consensus.cc:2454] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc5d744ba96e41c2ac6d16f9690c369e in term 0.
I20260806 20:29:11.089684 2547 raft_consensus.cc:2454] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc5d744ba96e41c2ac6d16f9690c369e in term 0.
I20260806 20:29:11.090368 2763 leader_election.cc:302] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b3dbc7b0bc8d466999992f2942f1d553, baeeb821d49c4305856834c1a1c65cc4, cc5d744ba96e41c2ac6d16f9690c369e; no voters:
I20260806 20:29:11.090849 2910 raft_consensus.cc:2781] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:29:11.090996 2910 raft_consensus.cc:482] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:29:11.091109 2910 raft_consensus.cc:3037] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 0 FOLLOWER]: Advancing to term 1
W20260806 20:29:11.091378 2600 tablet.cc:2369] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:29:11.093348 2910 raft_consensus.cc:504] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:11.094003 2910 leader_election.cc:288] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [CANDIDATE]: Term 1 election: Requested vote from peers b3dbc7b0bc8d466999992f2942f1d553 (127.17.109.193:37011), baeeb821d49c4305856834c1a1c65cc4 (127.17.109.194:44361)
I20260806 20:29:11.094499 2547 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb26bc76778489b9042b8ee1761b16a" candidate_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dbc7b0bc8d466999992f2942f1d553"
I20260806 20:29:11.094537 2695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "edb26bc76778489b9042b8ee1761b16a" candidate_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baeeb821d49c4305856834c1a1c65cc4"
I20260806 20:29:11.094727 2547 raft_consensus.cc:3037] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:11.094728 2695 raft_consensus.cc:3037] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:11.098035 2547 raft_consensus.cc:2454] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc5d744ba96e41c2ac6d16f9690c369e in term 1.
I20260806 20:29:11.098035 2695 raft_consensus.cc:2454] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc5d744ba96e41c2ac6d16f9690c369e in term 1.
I20260806 20:29:11.098512 2762 leader_election.cc:302] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3dbc7b0bc8d466999992f2942f1d553, cc5d744ba96e41c2ac6d16f9690c369e; no voters:
I20260806 20:29:11.098866 2910 raft_consensus.cc:2781] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:29:11.099344 2910 raft_consensus.cc:686] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 1 LEADER]: Becoming Leader. State: Replica: cc5d744ba96e41c2ac6d16f9690c369e, State: Running, Role: LEADER
I20260806 20:29:11.099871 2910 consensus_queue.cc:237] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } }
I20260806 20:29:11.104729 2388 catalog_manager.cc:5495] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc5d744ba96e41c2ac6d16f9690c369e (127.17.109.195). New cstate: current_term: 1 leader_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc5d744ba96e41c2ac6d16f9690c369e" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260806 20:29:11.156049 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:11.151056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41664 (local address 127.17.109.195:39023)
0806 20:29:11.151262 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:11.151267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:11.151284 (+ 17us) server_negotiation.cc:407] Connection header received
0806 20:29:11.151497 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:11.151501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:11.151610 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:11.151733 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:29:11.152038 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.152740 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.154084 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.154460 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.154547 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:11.154980 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:29:11.155446 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:29:11.155620 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:11.155682 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 20:29:11.155745 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 20:29:11.164683 2547 raft_consensus.cc:1261] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 1 FOLLOWER]: Refusing update from remote peer cc5d744ba96e41c2ac6d16f9690c369e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:29:11.164683 2695 raft_consensus.cc:1261] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 1 FOLLOWER]: Refusing update from remote peer cc5d744ba96e41c2ac6d16f9690c369e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:29:11.165824 2913 consensus_queue.cc:1035] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [LEADER]: Connected to new peer: Peer: permanent_uuid: "baeeb821d49c4305856834c1a1c65cc4" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:29:11.166204 2910 consensus_queue.cc:1035] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3dbc7b0bc8d466999992f2942f1d553" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:29:11.176771 2915 mvcc.cc:205] Tried to move back new op lower bound from 7315653227159924736 to 7315653226909466624. Current Snapshot: MvccSnapshot[applied={T|T < 7315653227159924736}]
I20260806 20:29:11.182430 2916 mvcc.cc:205] Tried to move back new op lower bound from 7315653227159924736 to 7315653226909466624. Current Snapshot: MvccSnapshot[applied={T|T < 7315653227159924736}]
I20260806 20:29:11.196408 2387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:29:11.196838 2387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:29:11.201400 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Sending DeleteTablet for 3 replicas of tablet edb26bc76778489b9042b8ee1761b16a
I20260806 20:29:11.202535 2527 tablet_service.cc:1514] Processing DeleteTablet for tablet edb26bc76778489b9042b8ee1761b16a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:11 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:54268
I20260806 20:29:11.202836 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 2431
I20260806 20:29:11.203641 2926 tablet_replica.cc:330] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: stopping tablet replica
I20260806 20:29:11.203718 2675 tablet_service.cc:1514] Processing DeleteTablet for tablet edb26bc76778489b9042b8ee1761b16a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:11 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:36694
I20260806 20:29:11.204159 2926 raft_consensus.cc:2229] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:29:11.204706 2926 raft_consensus.cc:2258] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:11.205947 2814 tablet_service.cc:1514] Processing DeleteTablet for tablet edb26bc76778489b9042b8ee1761b16a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:11 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:41662
I20260806 20:29:11.211047 2926 ts_tablet_manager.cc:1905] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:29:11.214785 2927 tablet_replica.cc:330] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: stopping tablet replica
I20260806 20:29:11.215394 2927 raft_consensus.cc:2229] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:29:11.216094 2927 raft_consensus.cc:2258] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:11.218776 2927 ts_tablet_manager.cc:1905] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:29:11.223158 2926 ts_tablet_manager.cc:1918] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:29:11.225364 2926 log.cc:1196] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/edb26bc76778489b9042b8ee1761b16a
I20260806 20:29:11.225519 2928 tablet_replica.cc:330] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: stopping tablet replica
I20260806 20:29:11.226004 2926 ts_tablet_manager.cc:1939] T edb26bc76778489b9042b8ee1761b16a P b3dbc7b0bc8d466999992f2942f1d553: Deleting consensus metadata
I20260806 20:29:11.226094 2928 raft_consensus.cc:2229] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:29:11.226372 2927 ts_tablet_manager.cc:1918] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:29:11.226598 2927 log.cc:1196] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/edb26bc76778489b9042b8ee1761b16a
I20260806 20:29:11.226946 2928 raft_consensus.cc:2258] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:11.227216 2927 ts_tablet_manager.cc:1939] T edb26bc76778489b9042b8ee1761b16a P baeeb821d49c4305856834c1a1c65cc4: Deleting consensus metadata
I20260806 20:29:11.223626 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:11.204400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.17.109.193:37011)
0806 20:29:11.204592 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:11.204598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:11.204618 (+ 20us) server_negotiation.cc:407] Connection header received
0806 20:29:11.204674 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:11.204681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:11.204782 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:11.204891 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:11.206946 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.208099 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.210076 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.210722 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.210855 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:11.214580 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:11.214601 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:11.214619 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:11.214669 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:11.219112 (+ 4443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:11.220086 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:11.220095 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:11.220099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:11.220285 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:11.220790 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:11.220795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:11.220798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:11.221349 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:11.221709 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:11.222950 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0806 20:29:11.223205 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 20:29:11.228380 2372 catalog_manager.cc:4841] TS b3dbc7b0bc8d466999992f2942f1d553 (127.17.109.193:37011): tablet edb26bc76778489b9042b8ee1761b16a (table test-table [id=5c1136697b994720a2a3c4af1ceaa774]) successfully deleted
I20260806 20:29:11.228855 2594 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:11.230232 2928 ts_tablet_manager.cc:1905] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:29:11.234074 2373 catalog_manager.cc:4841] TS baeeb821d49c4305856834c1a1c65cc4 (127.17.109.194:44361): tablet edb26bc76778489b9042b8ee1761b16a (table test-table [id=5c1136697b994720a2a3c4af1ceaa774]) successfully deleted
I20260806 20:29:11.238206 2928 ts_tablet_manager.cc:1918] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:29:11.238425 2928 log.cc:1196] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/edb26bc76778489b9042b8ee1761b16a
I20260806 20:29:11.239038 2928 ts_tablet_manager.cc:1939] T edb26bc76778489b9042b8ee1761b16a P cc5d744ba96e41c2ac6d16f9690c369e: Deleting consensus metadata
I20260806 20:29:11.241024 2373 catalog_manager.cc:4841] TS cc5d744ba96e41c2ac6d16f9690c369e (127.17.109.195:39023): tablet edb26bc76778489b9042b8ee1761b16a (table test-table [id=5c1136697b994720a2a3c4af1ceaa774]) successfully deleted
I20260806 20:29:11.336499 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 2431
I20260806 20:29:11.364919 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 2603
I20260806 20:29:11.381657 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:11.365948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.17.109.194:44361)
0806 20:29:11.366172 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:11.366186 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:11.366209 (+ 23us) server_negotiation.cc:407] Connection header received
0806 20:29:11.366279 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:11.366285 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:11.366377 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:11.366508 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:11.368315 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.369553 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.371020 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.371711 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.371843 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:11.374638 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:11.374658 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:11.374679 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:11.374757 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:11.378639 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:11.379305 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:11.379315 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:11.379319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:11.379477 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:11.379974 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:11.379979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:11.379983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:11.380469 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:11.380768 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:11.381043 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 20:29:11.381282 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260806 20:29:11.382498 2738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:11.480408 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 2603
I20260806 20:29:11.505929 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 2746
I20260806 20:29:11.522583 2899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:11.506852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.17.109.195:39023)
0806 20:29:11.507083 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:11.507090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:11.507129 (+ 39us) server_negotiation.cc:407] Connection header received
0806 20:29:11.507211 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:11.507217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:11.507314 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:11.507430 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:11.509000 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.510282 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.511874 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:11.512573 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:11.512726 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:11.515382 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:11.515399 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:11.515422 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:11.515474 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:11.519666 (+ 4192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:11.520329 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:11.520338 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:11.520341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:11.520539 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:11.520911 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:11.520917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:11.520920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:11.521419 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:11.521736 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:11.521908 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 20:29:11.522146 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 20:29:11.523456 2879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:11.646474 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 2746
I20260806 20:29:11.670971 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 2353
I20260806 20:29:11.672345 2422 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:11.787814 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 2353
I20260806 20:29:31.813820 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46883
--webserver_interface=127.17.109.254
--webserver_port=42853
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:32.170845 2934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:32.171190 2934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:32.171351 2934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:32.182791 2934 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 20:29:32.182926 2934 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 20:29:32.182991 2934 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 20:29:32.183069 2934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:29:32.183159 2934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:32.183252 2934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:32.183300 2934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:29:32.183355 2934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:29:32.193583 2934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=42853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:32.194860 2934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:32.196255 2934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:32.208847 2939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:32.208871 2940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:32.208755 2942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:32.209491 2934 server_base.cc:1034] running on GCE node
I20260806 20:29:32.210371 2934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:32.211869 2934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:32.213117 2934 hybrid_clock.cc:648] HybridClock initialized: now 1786048172213056 us; error 34 us; skew 500 ppm
I20260806 20:29:32.213500 2934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:32 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048172, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:32.226174 2934 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:29:32.228057 2934 webserver.cc:460] Webserver started at http://127.17.109.254:42853/ using document root <none> and password file <none>
I20260806 20:29:32.228870 2934 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:32.229029 2934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:32.235252 2934 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 20:29:32.238623 2950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:32.240217 2934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:32.240437 2934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "a1f104883451419d8828e780a72e58bc"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "5bc853282a857b4269b0d8d62b47a62f"
server_key_iv: "263bb76a10a91476f179bb301f672efa"
server_key_version: "encryptionkey@0"
I20260806 20:29:32.241581 2934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:32.267099 2934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:32.271790 2934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:32.272135 2934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:32.293187 2934 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:46883
I20260806 20:29:32.293241 3007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:46883 every 8 connection(s)
I20260806 20:29:32.294919 2934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:29:32.295766 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 2934
I20260806 20:29:32.296887 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:37011
--local_ip_for_outbound_sockets=127.17.109.193
--tserver_master_addrs=127.17.109.254:46883
--webserver_port=39391
--webserver_interface=127.17.109.193
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:29:32.302788 3008 sys_catalog.cc:263] Verifying existing consensus state
I20260806 20:29:32.306340 3008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Bootstrap starting.
I20260806 20:29:32.329064 3008 log.cc:824] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:32.347363 3008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 20:29:32.348088 3008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Bootstrap complete.
I20260806 20:29:32.358628 3008 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:29:32.359735 3008 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f104883451419d8828e780a72e58bc, State: Initialized, Role: FOLLOWER
I20260806 20:29:32.360409 3008 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:29:32.360688 3008 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:29:32.360834 3008 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:29:32.361038 3008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 2 FOLLOWER]: Advancing to term 3
I20260806 20:29:32.365726 3008 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:29:32.366273 3008 leader_election.cc:302] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1f104883451419d8828e780a72e58bc; no voters:
I20260806 20:29:32.367065 3008 leader_election.cc:288] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 20:29:32.367228 3012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 3 FOLLOWER]: Leader election won for term 3
I20260806 20:29:32.368498 3012 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [term 3 LEADER]: Becoming Leader. State: Replica: a1f104883451419d8828e780a72e58bc, State: Running, Role: LEADER
I20260806 20:29:32.369200 3012 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } }
I20260806 20:29:32.369557 3008 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:29:32.371934 3014 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1f104883451419d8828e780a72e58bc. Latest consensus state: current_term: 3 leader_uuid: "a1f104883451419d8828e780a72e58bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } } }
I20260806 20:29:32.372327 3014 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:32.372720 3013 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a1f104883451419d8828e780a72e58bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f104883451419d8828e780a72e58bc" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46883 } } }
I20260806 20:29:32.373044 3013 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:32.381453 3018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:29:32.393127 3018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:29:32.395493 3018 catalog_manager.cc:1259] Loaded cluster ID: 0e7bc51c390243099d1947ecb3c0dfa6
I20260806 20:29:32.395756 3018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:29:32.403105 3018 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:29:32.405705 3018 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Loaded TSK: 0
I20260806 20:29:32.406057 3018 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a1f104883451419d8828e780a72e58bc: Loaded TSK: 1
F20260806 20:29:32.407823 3018 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
I20260806 20:29:32.410400 3029 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f5b98d7e9ad google::LogMessage::Fail() at ??:0
@ 0x7f5b98d8188a google::LogMessage::SendToLog() at ??:0
@ 0x7f5b98d7e4de google::LogMessage::Flush() at ??:0
@ 0x7f5b98d7fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5b9cae27d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5b9cad173e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f5bae7b11c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f5bae7b0783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f5bae83c2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f5bae83c0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f5bae8865e9 std::function<>::operator()() at ??:0
@ 0x7f5bae7b4cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5bae7b2c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5bae83a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f5bae83a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5b9b9ca345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5b9b9cd91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f5b9b9cd799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5b9b9a8eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5b9ec826db start_thread at ??:0
@ 0x7f5b97c1861f clone at ??:0
W20260806 20:29:32.700007 3010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:32.700368 3010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:32.700546 3010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:32.708940 3010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:32.709137 3010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:32.709329 3010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:29:32.719914 3010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:37011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=39391
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:32.721351 3010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:32.722858 3010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:32.736963 3057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:32.737104 3058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:32.737066 3060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:32.738478 3010 server_base.cc:1034] running on GCE node
I20260806 20:29:32.739143 3010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:32.740303 3010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:32.741576 3010 hybrid_clock.cc:648] HybridClock initialized: now 1786048172741524 us; error 41 us; skew 500 ppm
I20260806 20:29:32.742010 3010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:32 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048172, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:32.745556 3010 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:29:32.747295 3010 webserver.cc:460] Webserver started at http://127.17.109.193:39391/ using document root <none> and password file <none>
I20260806 20:29:32.748229 3010 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:32.748395 3010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:32.754385 3010 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 20:29:32.757431 3068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:32.759138 3010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:29:32.759332 3010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "b3dbc7b0bc8d466999992f2942f1d553"
format_stamp: "Formatted at 2026-08-06 20:28:46 on dist-test-slave-6k22"
server_key: "f57e35a9eb53e5747f08b8bfc6cc7f3c"
server_key_iv: "0cf1a719207111e08d2476ec23bca448"
server_key_version: "encryptionkey@0"
I20260806 20:29:32.760509 3010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:32.806151 3010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:32.810650 3010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:32.810953 3010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:32.812063 3010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:32.813519 3010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:32.813656 3010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:32.813829 3010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:32.813908 3010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:32.850960 3010 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:37011
I20260806 20:29:32.851063 3194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:37011 every 8 connection(s)
I20260806 20:29:32.852481 3010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:29:32.857758 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:32.855714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:57959)
0806 20:29:32.856568 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:32.856836 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":364,"threads_started":1}
W20260806 20:29:32.858388 3195 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:46883 (0 consecutive failures): Network error: Failed to ping master at 127.17.109.254:46883: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
I20260806 20:29:32.859225 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:32.858894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:44115)
0806 20:29:32.859044 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:32.859122 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 20:29:32.860152 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:32.859781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:34319)
0806 20:29:32.859923 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:32.860012 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 20:29:32.860455 3195 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 20:29:32.860770 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 3010
I20260806 20:29:32.861394 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:44361
--local_ip_for_outbound_sockets=127.17.109.194
--tserver_master_addrs=127.17.109.254:46883
--webserver_port=34881
--webserver_interface=127.17.109.194
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:33.225230 3198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:33.225517 3198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:33.225685 3198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:33.233081 3198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:33.233201 3198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:33.233382 3198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:29:33.243599 3198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:44361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=34881
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:33.245138 3198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:33.246532 3198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:33.260341 3204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:33.260362 3203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:33.260452 3206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:33.261325 3198 server_base.cc:1034] running on GCE node
I20260806 20:29:33.262068 3198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:33.263303 3198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:33.264562 3198 hybrid_clock.cc:648] HybridClock initialized: now 1786048173264462 us; error 84 us; skew 500 ppm
I20260806 20:29:33.264957 3198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:33 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048173, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:33.269506 3198 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:29:33.271025 3198 webserver.cc:460] Webserver started at http://127.17.109.194:34881/ using document root <none> and password file <none>
I20260806 20:29:33.271809 3198 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:33.271961 3198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:33.277627 3198 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 20:29:33.280949 3214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:33.282433 3198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:29:33.282639 3198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "baeeb821d49c4305856834c1a1c65cc4"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "8d39b7904361fbcc86caf5335bf095f9"
server_key_iv: "cdc46a23e071b1f8a76535ff5174d16f"
server_key_version: "encryptionkey@0"
I20260806 20:29:33.283766 3198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:33.316491 3198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:33.321106 3198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:33.321429 3198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:33.322634 3198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:33.324203 3198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:33.324343 3198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:33.324501 3198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:33.324602 3198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:33.364144 3198 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:44361
I20260806 20:29:33.364287 3338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:44361 every 8 connection(s)
I20260806 20:29:33.365907 3198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:29:33.371454 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:33.369382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:46337)
0806 20:29:33.370250 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:33.370515 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":330,"threads_started":1}
W20260806 20:29:33.372231 3339 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:46883 (0 consecutive failures): Network error: Failed to ping master at 127.17.109.254:46883: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
I20260806 20:29:33.373183 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:33.372810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:42301)
0806 20:29:33.372987 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:33.373066 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 20:29:33.373870 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 3198
I20260806 20:29:33.374338 3341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:33.373979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:38375)
0806 20:29:33.374120 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:33.374220 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 20:29:33.374600 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:39023
--local_ip_for_outbound_sockets=127.17.109.195
--tserver_master_addrs=127.17.109.254:46883
--webserver_port=36177
--webserver_interface=127.17.109.195
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:33.374670 3339 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 20:29:33.712579 3343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:33.712908 3343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:33.713085 3343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:33.721311 3343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:33.721455 3343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:33.721673 3343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:29:33.732672 3343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:39023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=36177
--tserver_master_addrs=127.17.109.254:46883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:33.734125 3343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:33.735608 3343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:33.750286 3352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:33.750327 3349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:33.750204 3348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:33.751307 3343 server_base.cc:1034] running on GCE node
I20260806 20:29:33.752054 3343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:33.753113 3343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:33.754367 3343 hybrid_clock.cc:648] HybridClock initialized: now 1786048173754329 us; error 71 us; skew 500 ppm
I20260806 20:29:33.754729 3343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:33 dist-test-slave-6k22 krb5kdc[1751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048173, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:33.758080 3343 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:29:33.759608 3343 webserver.cc:460] Webserver started at http://127.17.109.195:36177/ using document root <none> and password file <none>
I20260806 20:29:33.760300 3343 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:33.760407 3343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:33.766830 3343 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 20:29:33.769719 3360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:33.771117 3343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:29:33.771287 3343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "cc5d744ba96e41c2ac6d16f9690c369e"
format_stamp: "Formatted at 2026-08-06 20:28:47 on dist-test-slave-6k22"
server_key: "f2617d83b01f21f24200556e7997ed8b"
server_key_iv: "b43001dc58623956401ea960e4cbe53e"
server_key_version: "encryptionkey@0"
I20260806 20:29:33.772306 3343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:33.787446 3343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:33.791735 3343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:33.792027 3343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:33.793043 3343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:33.794368 3343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:33.794456 3343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:33.794595 3343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:33.794665 3343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:33.830575 3343 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:39023
I20260806 20:29:33.830710 3481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:39023 every 8 connection(s)
I20260806 20:29:33.832114 3343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:29:33.836169 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 3343
I20260806 20:29:33.837288 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:33.835240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:37343)
0806 20:29:33.836070 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:33.836373 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":381,"threads_started":1}
W20260806 20:29:33.838172 3482 heartbeater.cc:644] Failed to heartbeat to 127.17.109.254:46883 (0 consecutive failures): Network error: Failed to ping master at 127.17.109.254:46883: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
I20260806 20:29:33.839109 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:33.838750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:45323)
0806 20:29:33.838914 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:33.838998 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 20:29:33.840268 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:33.839866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:47087)
0806 20:29:33.840026 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:33.840125 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260806 20:29:33.840536 3482 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 20:29:33.862020 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:33.860934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:37243)
0806 20:29:33.861434 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:33.861588 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":245,"threads_started":1}
I20260806 20:29:34.376401 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:34.375348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:57767)
0806 20:29:34.376040 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:34.376138 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":395,"threads_started":1}
I20260806 20:29:34.842084 3488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:34.841130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:35345)
0806 20:29:34.841632 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:34.841811 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":249,"threads_started":1}
I20260806 20:29:34.863931 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:34.863093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:38081)
0806 20:29:34.863670 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:34.863760 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":332,"threads_started":1}
I20260806 20:29:35.378185 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:35.377384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:34671)
0806 20:29:35.377905 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:35.377999 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":289,"threads_started":1}
I20260806 20:29:35.844010 3491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:35.843059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:32853)
0806 20:29:35.843677 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:35.843816 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":270,"threads_started":1}
I20260806 20:29:35.865545 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:35.864774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:36989)
0806 20:29:35.865274 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:35.865366 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":296,"threads_started":1}
I20260806 20:29:36.380191 3493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:36.379361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:57357)
0806 20:29:36.379915 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:36.380014 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":349,"threads_started":1}
I20260806 20:29:36.845875 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:36.844935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:50429)
0806 20:29:36.845453 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:36.845543 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":279,"threads_started":1}
I20260806 20:29:36.867753 3495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:36.866463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:44681)
0806 20:29:36.867317 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:36.867417 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":560,"threads_started":1}
I20260806 20:29:37.381924 3496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:37.381125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:42795)
0806 20:29:37.381632 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:37.381734 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":248,"threads_started":1}
I20260806 20:29:37.847865 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:37.846994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:47095)
0806 20:29:37.847450 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:37.847536 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":225,"threads_started":1}
I20260806 20:29:37.869625 3498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:37.868737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:41149)
0806 20:29:37.869348 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:37.869436 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":379,"threads_started":1}
I20260806 20:29:38.383973 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:38.383040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:46879)
0806 20:29:38.383680 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:38.383784 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":278,"threads_started":1}
I20260806 20:29:38.849838 3500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:38.849029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:47201)
0806 20:29:38.849562 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:38.849651 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":282,"threads_started":1}
I20260806 20:29:38.871408 3501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:38.870580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:54911)
0806 20:29:38.871145 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:38.871244 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":328,"threads_started":1}
I20260806 20:29:39.385773 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:39.384911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:37313)
0806 20:29:39.385424 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:39.385532 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":261,"threads_started":1}
I20260806 20:29:39.851763 3503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:39.850809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:59979)
0806 20:29:39.851404 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:39.851506 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":293,"threads_started":1}
I20260806 20:29:39.873232 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:39.872421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:48015)
0806 20:29:39.872934 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:39.873021 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":221,"threads_started":1}
I20260806 20:29:40.387984 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:40.386893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:36973)
0806 20:29:40.387509 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:40.387651 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":257,"threads_started":1}
I20260806 20:29:40.853830 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:40.852814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:44003)
0806 20:29:40.853401 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:40.853517 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":235,"threads_started":1}
I20260806 20:29:40.875627 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:40.874427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:49863)
0806 20:29:40.875268 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:40.875375 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":419,"threads_started":1}
I20260806 20:29:41.389827 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:41.389016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:52895)
0806 20:29:41.389534 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:41.389628 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":266,"threads_started":1}
I20260806 20:29:41.855799 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:41.854878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:34199)
0806 20:29:41.855480 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:41.855619 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":356,"threads_started":1}
I20260806 20:29:41.877592 3510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:41.876793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:36565)
0806 20:29:41.877304 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:41.877401 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":227,"threads_started":1}
I20260806 20:29:42.391538 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:42.390704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:37079)
0806 20:29:42.391228 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:42.391351 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":246,"threads_started":1}
I20260806 20:29:42.857525 3512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:42.856763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:56045)
0806 20:29:42.857280 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:42.857359 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":257,"threads_started":1}
I20260806 20:29:42.879518 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:42.878713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:36077)
0806 20:29:42.879273 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:42.879355 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":340,"threads_started":1}
I20260806 20:29:43.393545 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:43.392712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:54691)
0806 20:29:43.393250 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:43.393350 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":280,"threads_started":1}
I20260806 20:29:43.859493 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:43.858597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:44851)
0806 20:29:43.859151 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:43.859270 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":319,"threads_started":1}
I20260806 20:29:43.881486 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:43.880556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:45531)
0806 20:29:43.881161 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:43.881283 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":313,"threads_started":1}
I20260806 20:29:44.395444 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:44.394609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:39929)
0806 20:29:44.395133 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:44.395245 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":266,"threads_started":1}
I20260806 20:29:44.861305 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:44.860554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:53467)
0806 20:29:44.861031 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:44.861123 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":262,"threads_started":1}
I20260806 20:29:44.883306 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:44.882556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:57255)
0806 20:29:44.883024 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:44.883113 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":251,"threads_started":1}
I20260806 20:29:45.397269 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:45.396500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:33719)
0806 20:29:45.397020 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:45.397105 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":284,"threads_started":1}
I20260806 20:29:45.863236 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:45.862310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:46587)
0806 20:29:45.862906 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:45.862989 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":298,"threads_started":1}
I20260806 20:29:45.885299 3526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:45.884324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:41007)
0806 20:29:45.884994 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:45.885098 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":419,"threads_started":1}
I20260806 20:29:46.399410 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:46.398423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:49583)
0806 20:29:46.398995 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:46.399177 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":292,"threads_started":1}
I20260806 20:29:46.865314 3529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:46.864377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:47845)
0806 20:29:46.864946 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:46.865045 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":304,"threads_started":1}
I20260806 20:29:46.887246 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:46.886319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:50591)
0806 20:29:46.886867 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:46.887001 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":231,"threads_started":1}
I20260806 20:29:47.401394 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:47.400532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:51513)
0806 20:29:47.401076 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:47.401188 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":290,"threads_started":1}
I20260806 20:29:47.867146 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:47.866344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:45761)
0806 20:29:47.866866 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:47.866957 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":236,"threads_started":1}
I20260806 20:29:47.889355 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:47.888563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:54567)
0806 20:29:47.889060 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:47.889166 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":270,"threads_started":1}
I20260806 20:29:48.403820 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:48.402758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.194:36821)
0806 20:29:48.403393 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:48.403522 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":345,"threads_started":1}
I20260806 20:29:48.846312 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 3010
I20260806 20:29:48.867208 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:48.847637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48606 (local address 127.17.109.193:37011)
0806 20:29:48.848496 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:48.848518 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:48.848574 (+ 56us) server_negotiation.cc:407] Connection header received
0806 20:29:48.850342 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:48.850352 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:48.850791 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:48.851093 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:48.852914 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:48.854984 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:48.856443 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:48.857145 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:48.857294 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:48.860270 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:48.860279 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:48.860297 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:48.860342 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:48.863435 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:48.864382 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:48.864389 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:48.864397 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:48.864602 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:48.865077 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:48.865084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:48.865087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:48.865700 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:48.865971 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:48.866404 (+ 433us) server_negotiation.cc:299] Negotiation successful
0806 20:29:48.866880 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":357,"threads_started":1}
I20260806 20:29:48.868846 3188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:48.869220 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:48.868330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.195:56389)
0806 20:29:48.868886 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:48.869020 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":348,"threads_started":1}
I20260806 20:29:48.956754 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 3010
I20260806 20:29:48.956851 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:48.955387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46883 (local address 127.17.109.193:46659)
0806 20:29:48.956109 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:48.956393 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.254:46883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":329,"threads_started":1}
I20260806 20:29:48.977638 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 3198
I20260806 20:29:48.997234 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:48.978662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.17.109.194:44361)
0806 20:29:48.979200 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:48.979215 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:48.979255 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:29:48.980921 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:48.980933 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:48.981423 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:48.981835 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:48.983462 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:48.984891 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:48.986190 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:48.986998 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:48.987209 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:48.990249 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:48.990258 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:48.990276 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:48.990321 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:48.993870 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:48.994585 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:48.994593 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:48.994600 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:48.994790 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:48.995122 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:48.995126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:48.995128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:48.995761 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:48.996044 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:48.996529 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 20:29:48.996911 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":234,"threads_started":1}
I20260806 20:29:48.998703 3334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:49.086226 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 3198
I20260806 20:29:49.107309 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 3343
I20260806 20:29:49.127012 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:49.108720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.17.109.195:39023)
0806 20:29:49.109402 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:49.109417 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:49.109461 (+ 44us) server_negotiation.cc:407] Connection header received
0806 20:29:49.111208 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:49.111217 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:49.111790 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:49.112107 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:49.113725 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:49.114946 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:49.116416 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:49.117195 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:49.117335 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:49.120299 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:49.120307 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:49.120324 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:49.120363 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:49.123491 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:49.124392 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:49.124399 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:49.124406 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:49.124605 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:49.125041 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:49.125045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:49.125048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:49.125606 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:49.125852 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:49.126309 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 20:29:49.126649 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":324,"threads_started":1}
I20260806 20:29:49.128464 3477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:49.214255 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 3343
2026-08-06T20:29:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65017 ms)
[----------] 30 tests from SecurityITest (310194 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:29:49 dist-test-slave-6k22 krb5kdc[3547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:29:49 dist-test-slave-6k22 krb5kdc[3547](info): set up 2 sockets
Aug 06 20:29:49 dist-test-slave-6k22 krb5kdc[3547](info): commencing operation
krb5kdc: starting...
W20260806 20:29:50.410341 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.006s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:29:50 dist-test-slave-6k22 krb5kdc[3547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:29:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:29:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:29:50.744539 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:35941
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:51.088114 3560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:51.088487 3560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:51.088660 3560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:51.096215 3560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:29:51.096375 3560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:51.096493 3560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:51.096590 3560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:29:51.096673 3560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:29:51.106902 3560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:35941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:51.108371 3560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:51.109790 3560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:51.121944 3565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:51.121904 3568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:51.121994 3566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:51.123283 3560 server_base.cc:1034] running on GCE node
I20260806 20:29:51.124158 3560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:51.125638 3560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:51.126842 3560 hybrid_clock.cc:648] HybridClock initialized: now 1786048191126793 us; error 33 us; skew 500 ppm
I20260806 20:29:51.127211 3560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:51 dist-test-slave-6k22 krb5kdc[3547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048191, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:51.138255 3560 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:29:51.140038 3560 webserver.cc:460] Webserver started at http://127.17.109.254:33615/ using document root <none> and password file <none>
I20260806 20:29:51.140782 3560 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:51.140894 3560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:51.141192 3560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:51.144625 3560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "8a7637c6cfe8481490902622b7f3c63f"
format_stamp: "Formatted at 2026-08-06 20:29:51 on dist-test-slave-6k22"
server_key: "9f3faea5574b749ac9d03bfe624d1124"
server_key_iv: "76eb96023890aecb584d23503f9f2dcf"
server_key_version: "encryptionkey@0"
I20260806 20:29:51.145439 3560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "8a7637c6cfe8481490902622b7f3c63f"
format_stamp: "Formatted at 2026-08-06 20:29:51 on dist-test-slave-6k22"
server_key: "9f3faea5574b749ac9d03bfe624d1124"
server_key_iv: "76eb96023890aecb584d23503f9f2dcf"
server_key_version: "encryptionkey@0"
I20260806 20:29:51.151036 3560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:29:51.154768 3576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:51.156256 3560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:51.156411 3560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "8a7637c6cfe8481490902622b7f3c63f"
format_stamp: "Formatted at 2026-08-06 20:29:51 on dist-test-slave-6k22"
server_key: "9f3faea5574b749ac9d03bfe624d1124"
server_key_iv: "76eb96023890aecb584d23503f9f2dcf"
server_key_version: "encryptionkey@0"
I20260806 20:29:51.156574 3560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:51.172276 3560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:51.176247 3560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:51.176483 3560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:51.196646 3560 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:35941
I20260806 20:29:51.196686 3631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:35941 every 8 connection(s)
I20260806 20:29:51.198005 3560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:29:51.201912 3632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:51.205837 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 3560
I20260806 20:29:51.206060 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:29:51.206691 17847 external_mini_cluster.cc:1428] Setting key b515848f7d615eb0e3fa11d448673b0e
I20260806 20:29:51.216602 3632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f: Bootstrap starting.
I20260806 20:29:51.220682 3632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:51.221788 3632 log.cc:824] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f: Log is configured to *not* fsync() on all Append() calls
Aug 06 20:29:51 dist-test-slave-6k22 krb5kdc[3547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:51.225204 3632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f: No bootstrap required, opened a new log
I20260806 20:29:51.234309 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:51.208790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60180 (local address 127.17.109.254:35941)
0806 20:29:51.209671 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:51.209689 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:51.209743 (+ 54us) server_negotiation.cc:407] Connection header received
0806 20:29:51.211808 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:51.211840 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:51.212405 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:51.212887 (+ 482us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:51.214217 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:51.216341 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:51.217762 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:51.219027 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:51.219245 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:51.224343 (+ 5098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:51.224356 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:51.224376 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:51.224423 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:51.229198 (+ 4775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:51.230015 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:51.230030 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:51.230039 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:51.230280 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:51.230713 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:51.230719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:51.230722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:51.231513 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:51.231914 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:51.232251 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 20:29:51.232781 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":375,"threads_started":1}
I20260806 20:29:51.236846 3632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7637c6cfe8481490902622b7f3c63f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35941 } }
I20260806 20:29:51.237273 3632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:51.237411 3632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a7637c6cfe8481490902622b7f3c63f, State: Initialized, Role: FOLLOWER
I20260806 20:29:51.238050 3632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7637c6cfe8481490902622b7f3c63f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35941 } }
I20260806 20:29:51.238289 3632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:29:51.238422 3632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:29:51.238554 3632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:51.241020 3632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7637c6cfe8481490902622b7f3c63f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35941 } }
I20260806 20:29:51.241588 3632 leader_election.cc:302] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a7637c6cfe8481490902622b7f3c63f; no voters:
I20260806 20:29:51.242261 3632 leader_election.cc:288] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:29:51.242422 3637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:29:51.243257 3637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [term 1 LEADER]: Becoming Leader. State: Replica: 8a7637c6cfe8481490902622b7f3c63f, State: Running, Role: LEADER
I20260806 20:29:51.243896 3637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7637c6cfe8481490902622b7f3c63f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35941 } }
I20260806 20:29:51.244616 3632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:29:51.248317 3638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a7637c6cfe8481490902622b7f3c63f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7637c6cfe8481490902622b7f3c63f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35941 } } }
I20260806 20:29:51.248935 3638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:51.248333 3639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a7637c6cfe8481490902622b7f3c63f. Latest consensus state: current_term: 1 leader_uuid: "8a7637c6cfe8481490902622b7f3c63f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a7637c6cfe8481490902622b7f3c63f" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35941 } } }
I20260806 20:29:51.249614 3639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:51.251178 3648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:29:51.257217 3648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:29:51.267215 3648 catalog_manager.cc:1347] Generated new cluster ID: 4459649437ec4e4e8383cd589afae983
I20260806 20:29:51.267346 3648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:29:51.298260 3648 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:29:51.300503 3648 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:29:51.310477 3648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f: Generated new TSK 0
I20260806 20:29:51.311461 3648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:29:51.399863 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:35941
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:51.746177 3659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:51.746448 3659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:51.746627 3659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:51.754057 3659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:51.754169 3659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:51.754376 3659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:29:51.764520 3659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:35941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:51.765799 3659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:51.767122 3659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:51.780479 3665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:51.780489 3664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:51.780519 3668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:51.781319 3659 server_base.cc:1034] running on GCE node
I20260806 20:29:51.781960 3659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:51.782932 3659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:51.784170 3659 hybrid_clock.cc:648] HybridClock initialized: now 1786048191784113 us; error 39 us; skew 500 ppm
I20260806 20:29:51.784492 3659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:51 dist-test-slave-6k22 krb5kdc[3547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048191, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:51.787752 3659 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:29:51.789258 3659 webserver.cc:460] Webserver started at http://127.17.109.193:41525/ using document root <none> and password file <none>
I20260806 20:29:51.789937 3659 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:51.790083 3659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:51.790405 3659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:51.793490 3659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "77d62cb793934616932c794c2ad9404f"
format_stamp: "Formatted at 2026-08-06 20:29:51 on dist-test-slave-6k22"
server_key: "28bd30362236ac0a9449eb9ffcfbf761"
server_key_iv: "39004808d2a02641417645a0252aedae"
server_key_version: "encryptionkey@0"
I20260806 20:29:51.794283 3659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "77d62cb793934616932c794c2ad9404f"
format_stamp: "Formatted at 2026-08-06 20:29:51 on dist-test-slave-6k22"
server_key: "28bd30362236ac0a9449eb9ffcfbf761"
server_key_iv: "39004808d2a02641417645a0252aedae"
server_key_version: "encryptionkey@0"
I20260806 20:29:51.799644 3659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 20:29:51.803507 3676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:51.805006 3659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 20:29:51.805189 3659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "77d62cb793934616932c794c2ad9404f"
format_stamp: "Formatted at 2026-08-06 20:29:51 on dist-test-slave-6k22"
server_key: "28bd30362236ac0a9449eb9ffcfbf761"
server_key_iv: "39004808d2a02641417645a0252aedae"
server_key_version: "encryptionkey@0"
I20260806 20:29:51.805408 3659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:51.832978 3659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:51.837651 3659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:51.837994 3659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:51.839130 3659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:51.840682 3659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:51.840792 3659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:51.840955 3659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:51.841037 3659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:51.877696 3659 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:34663
I20260806 20:29:51.877828 3795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:34663 every 8 connection(s)
I20260806 20:29:51.879097 3659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:29:51.884682 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 3659
I20260806 20:29:51.884924 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:29:51.885849 17847 external_mini_cluster.cc:1428] Setting key 02971a1c081c8620be63c1b5d6d1dd4b
Aug 06 20:29:51 dist-test-slave-6k22 krb5kdc[3547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048191, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:51.903144 3798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:51.881859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:41913 (local address 127.17.109.254:35941)
0806 20:29:51.882328 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:51.882336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:51.883297 (+ 961us) server_negotiation.cc:407] Connection header received
0806 20:29:51.885920 (+ 2623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:51.885927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:51.886018 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:51.886110 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:51.887972 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:51.889072 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:51.891187 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:51.891925 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:51.892070 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:51.895946 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:51.895955 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:51.895962 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:51.896008 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:51.899626 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:51.900379 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:51.900387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:51.900393 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:51.900556 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:51.901256 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:51.901260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:51.901262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:51.901539 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:51.901661 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:51.902684 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0806 20:29:51.902881 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":208,"threads_started":1}
I20260806 20:29:51.904873 3799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:51.882412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:35941 (local address 127.17.109.193:41913)
0806 20:29:51.883126 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:51.883176 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:51.885542 (+ 2366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:51.886400 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:51.886412 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:51.886964 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:51.887809 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:51.887829 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:51.889268 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:51.889273 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:51.891042 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:51.891055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:51.892169 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:51.892176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:51.892446 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:51.893205 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:29:51.893230 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:51.895699 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:51.899771 (+ 4072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:51.899777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:51.899791 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:51.900248 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:51.900782 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:51.900790 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:51.900795 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:51.901090 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:51.901687 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:51.901698 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:51.902214 (+ 516us) client_negotiation.cc:769] Sending connection context
0806 20:29:51.902567 (+ 353us) client_negotiation.cc:240] Negotiation successful
0806 20:29:51.903248 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":266,"threads_started":1}
I20260806 20:29:51.906716 3796 heartbeater.cc:347] Connected to a master server at 127.17.109.254:35941
I20260806 20:29:51.907053 3796 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:51.907887 3796 heartbeater.cc:511] Master 127.17.109.254:35941 requested a full tablet report, sending...
I20260806 20:29:51.910064 3594 ts_manager.cc:194] Registered new tserver with Master: 77d62cb793934616932c794c2ad9404f (127.17.109.193:34663)
I20260806 20:29:51.911537 3594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:41913
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:29:51.958307 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:35941
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:52.313345 3802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:52.313622 3802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:52.313812 3802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:52.321225 3802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:52.321337 3802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:52.321542 3802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:29:52.331386 3802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:35941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:52.332798 3802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:52.334174 3802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:52.349279 3811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:52.349298 3808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:52.349331 3809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:52.350025 3802 server_base.cc:1034] running on GCE node
I20260806 20:29:52.350704 3802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:52.351838 3802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:52.353089 3802 hybrid_clock.cc:648] HybridClock initialized: now 1786048192353043 us; error 42 us; skew 500 ppm
I20260806 20:29:52.353421 3802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:52 dist-test-slave-6k22 krb5kdc[3547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048192, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:52.356700 3802 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:29:52.358108 3802 webserver.cc:460] Webserver started at http://127.17.109.194:39217/ using document root <none> and password file <none>
I20260806 20:29:52.358830 3802 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:52.358981 3802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:52.359298 3802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:52.362812 3802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "bb22fce8bf484312ab06296959095374"
format_stamp: "Formatted at 2026-08-06 20:29:52 on dist-test-slave-6k22"
server_key: "aa03aab07ec0011f72531a3408382047"
server_key_iv: "c4ba24e405938adb1228d732cbed57ce"
server_key_version: "encryptionkey@0"
I20260806 20:29:52.363652 3802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "bb22fce8bf484312ab06296959095374"
format_stamp: "Formatted at 2026-08-06 20:29:52 on dist-test-slave-6k22"
server_key: "aa03aab07ec0011f72531a3408382047"
server_key_iv: "c4ba24e405938adb1228d732cbed57ce"
server_key_version: "encryptionkey@0"
I20260806 20:29:52.369040 3802 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:29:52.372676 3819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:52.374148 3802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:29:52.374337 3802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "bb22fce8bf484312ab06296959095374"
format_stamp: "Formatted at 2026-08-06 20:29:52 on dist-test-slave-6k22"
server_key: "aa03aab07ec0011f72531a3408382047"
server_key_iv: "c4ba24e405938adb1228d732cbed57ce"
server_key_version: "encryptionkey@0"
I20260806 20:29:52.374543 3802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:52.417400 3802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:52.422178 3802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:52.422513 3802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:52.423615 3802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:52.425060 3802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:52.425133 3802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:52.425271 3802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:52.425323 3802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:52.460332 3802 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:32929
I20260806 20:29:52.460475 3945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:32929 every 8 connection(s)
I20260806 20:29:52.461853 3802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:29:52.471199 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 3802
I20260806 20:29:52.471436 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:29:52.471992 17847 external_mini_cluster.cc:1428] Setting key 8029809a54ea2b355879301e22120a6d
Aug 06 20:29:52 dist-test-slave-6k22 krb5kdc[3547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048192, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:52.483032 3948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:52.464528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:39351 (local address 127.17.109.254:35941)
0806 20:29:52.464970 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:52.464974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:52.466000 (+ 1026us) server_negotiation.cc:407] Connection header received
0806 20:29:52.468033 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:52.468039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:52.468132 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:52.468263 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:52.469846 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:52.470602 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:52.472385 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:52.473025 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:52.473167 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:52.476795 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:52.476812 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:52.476816 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:52.476850 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:52.479750 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:52.480543 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:52.480547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:52.480549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:52.480687 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:52.481161 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:52.481166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:52.481168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:52.481455 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:52.481617 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:52.482360 (+ 743us) server_negotiation.cc:299] Negotiation successful
0806 20:29:52.482617 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":225,"threads_started":1}
I20260806 20:29:52.484210 3950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:52.465099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:35941 (local address 127.17.109.194:39351)
0806 20:29:52.465826 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:52.465869 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:52.467760 (+ 1891us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:52.468506 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:52.468514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:52.469069 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:52.469647 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:52.469665 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:52.470781 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:52.470785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:52.472244 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:52.472257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:52.473259 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:52.473266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:52.473531 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:52.474276 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:29:52.474299 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:52.476547 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:52.479896 (+ 3349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:52.479906 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:52.479924 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:52.480389 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:52.480809 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:52.480815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:52.480819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:52.481027 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:52.481588 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:52.481597 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:52.481979 (+ 382us) client_negotiation.cc:769] Sending connection context
0806 20:29:52.482260 (+ 281us) client_negotiation.cc:240] Negotiation successful
0806 20:29:52.482874 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":279,"threads_started":1}
I20260806 20:29:52.485863 3946 heartbeater.cc:347] Connected to a master server at 127.17.109.254:35941
I20260806 20:29:52.486200 3946 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:52.486934 3946 heartbeater.cc:511] Master 127.17.109.254:35941 requested a full tablet report, sending...
I20260806 20:29:52.488478 3594 ts_manager.cc:194] Registered new tserver with Master: bb22fce8bf484312ab06296959095374 (127.17.109.194:32929)
I20260806 20:29:52.489356 3594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:39351
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:29:52.546696 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:35941
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:52.887275 3953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:52.887611 3953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:52.887784 3953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:52.895186 3953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:52.895350 3953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:52.895519 3953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:29:52.905611 3953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:35941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:52.906978 3953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:52.908350 3953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 20:29:52.914587 3796 heartbeater.cc:503] Master 127.17.109.254:35941 was elected leader, sending a full tablet report...
W20260806 20:29:52.922377 3959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:52.922400 3961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:52.922338 3958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:52.922655 3953 server_base.cc:1034] running on GCE node
I20260806 20:29:52.923533 3953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:52.924453 3953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:52.925655 3953 hybrid_clock.cc:648] HybridClock initialized: now 1786048192925561 us; error 78 us; skew 500 ppm
I20260806 20:29:52.925932 3953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:52 dist-test-slave-6k22 krb5kdc[3547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048192, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:52.929423 3953 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:29:52.931097 3953 webserver.cc:460] Webserver started at http://127.17.109.195:39751/ using document root <none> and password file <none>
I20260806 20:29:52.932018 3953 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:52.932191 3953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:52.932546 3953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:52.935792 3953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "9d093434dc57431a8944bc154c0ab55d"
format_stamp: "Formatted at 2026-08-06 20:29:52 on dist-test-slave-6k22"
server_key: "baca55dffc6d2de388d26d4872c761da"
server_key_iv: "66177735189d44b5353c6ec47e9820d4"
server_key_version: "encryptionkey@0"
I20260806 20:29:52.936604 3953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "9d093434dc57431a8944bc154c0ab55d"
format_stamp: "Formatted at 2026-08-06 20:29:52 on dist-test-slave-6k22"
server_key: "baca55dffc6d2de388d26d4872c761da"
server_key_iv: "66177735189d44b5353c6ec47e9820d4"
server_key_version: "encryptionkey@0"
I20260806 20:29:52.942368 3953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:29:52.946349 3970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:52.947875 3953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:52.948047 3953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "9d093434dc57431a8944bc154c0ab55d"
format_stamp: "Formatted at 2026-08-06 20:29:52 on dist-test-slave-6k22"
server_key: "baca55dffc6d2de388d26d4872c761da"
server_key_iv: "66177735189d44b5353c6ec47e9820d4"
server_key_version: "encryptionkey@0"
I20260806 20:29:52.948271 3953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:52.964800 3953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:52.969007 3953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:52.969312 3953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:52.970358 3953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:52.971786 3953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:52.971860 3953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:52.971988 3953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:52.972038 3953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:53.007032 3953 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:39165
I20260806 20:29:53.007035 4093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:39165 every 8 connection(s)
I20260806 20:29:53.008412 3953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:29:53.018756 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 3953
I20260806 20:29:53.018996 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:29:53.019523 17847 external_mini_cluster.cc:1428] Setting key 90e07ff5d64707c9a2f8476258ed4bf0
Aug 06 20:29:53 dist-test-slave-6k22 krb5kdc[3547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048192, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:53.030725 4096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.011392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:60943 (local address 127.17.109.254:35941)
0806 20:29:53.012046 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.012051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.012828 (+ 777us) server_negotiation.cc:407] Connection header received
0806 20:29:53.014841 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.014847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.014944 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.015072 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:53.016791 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.017726 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.019502 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.020230 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.020348 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.024930 (+ 4582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:53.024935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:53.024938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:53.024973 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:53.027840 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.028595 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.028599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.028601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.028741 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.029132 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.029135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.029137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.029409 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:53.029530 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.030263 (+ 733us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.030439 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":388,"threads_started":1}
I20260806 20:29:53.031806 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.011867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:35941 (local address 127.17.109.195:60943)
0806 20:29:53.012640 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:53.012691 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:53.014550 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:53.015343 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:53.015353 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:53.015993 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:53.016592 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.016611 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.017922 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.017927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.019341 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.019354 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.020486 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.020495 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.020726 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.021582 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:29:53.021607 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:53.024709 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:53.027999 (+ 3290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:53.028007 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:53.028021 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:53.028470 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:53.028838 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:53.028841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:53.028843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:53.029029 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:53.029518 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:53.029524 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:53.029908 (+ 384us) client_negotiation.cc:769] Sending connection context
0806 20:29:53.030139 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 20:29:53.030559 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":284,"threads_started":1}
I20260806 20:29:53.033362 4094 heartbeater.cc:347] Connected to a master server at 127.17.109.254:35941
I20260806 20:29:53.033751 4094 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:53.034766 4094 heartbeater.cc:511] Master 127.17.109.254:35941 requested a full tablet report, sending...
I20260806 20:29:53.036381 3594 ts_manager.cc:194] Registered new tserver with Master: 9d093434dc57431a8944bc154c0ab55d (127.17.109.195:39165)
I20260806 20:29:53.037216 3594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:60943
I20260806 20:29:53.048745 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:29:53.067973 4096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.054126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.17.109.254:35941)
0806 20:29:53.054390 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.054397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.054592 (+ 195us) server_negotiation.cc:407] Connection header received
0806 20:29:53.054870 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.054880 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.054947 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.055052 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:53.056860 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.057655 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.059079 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.059537 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.059728 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.062593 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:53.062598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:53.062601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:53.062635 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:53.065235 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.065919 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.065929 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.065931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.066051 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.066424 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.066432 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.066436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.066770 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:53.067030 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.067368 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.067635 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 20:29:53.072497 3594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:29:53.080459 3594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:29:53.104466 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.093866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:32929 (local address 127.0.0.1:48976)
0806 20:29:53.095002 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:53.095062 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:53.095314 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:53.096461 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:53.096468 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:53.096500 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:53.096934 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.096954 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.099017 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.099023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.101500 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.101515 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.103419 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.103427 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.103667 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.103714 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 20:29:53.103868 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 20:29:53.104072 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":481,"threads_started":1}
I20260806 20:29:53.104440 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.093995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.17.109.194:32929)
0806 20:29:53.095955 (+ 1960us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.095961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.095985 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:29:53.096061 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.096066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.096255 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.096454 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:53.097164 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.098868 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.101696 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.103201 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.103405 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.103619 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.103906 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.104084 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1808,"thread_start_us":1702,"threads_started":1}
I20260806 20:29:53.105700 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.094295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:39165 (local address 127.0.0.1:59384)
0806 20:29:53.097149 (+ 2854us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:53.097207 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:53.097331 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:53.098282 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:53.098288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:53.098315 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:53.098765 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.098776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.100834 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.100840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.102896 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.102910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.104861 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.104869 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.105068 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.105119 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 20:29:53.105213 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 20:29:53.105371 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2728,"thread_start_us":3039,"threads_started":1}
I20260806 20:29:53.105917 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.094791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59384 (local address 127.17.109.195:39165)
0806 20:29:53.097144 (+ 2353us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.097152 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.097474 (+ 322us) server_negotiation.cc:407] Connection header received
0806 20:29:53.097577 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.097586 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.097828 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.098010 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:53.098994 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.100617 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.103081 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.104608 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.104797 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.104971 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.105400 (+ 429us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.105574 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2163,"thread_start_us":2392,"threads_started":1}
I20260806 20:29:53.107980 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.096893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:34663 (local address 127.0.0.1:45508)
0806 20:29:53.099436 (+ 2543us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:53.099470 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:53.099640 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:53.100764 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:53.100769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:53.100795 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:53.101270 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.101280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.103209 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.103215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.105385 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.105399 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.107250 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.107257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.107407 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.107429 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 20:29:53.107482 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 20:29:53.107618 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"thread_start_us":764,"threads_started":1}
I20260806 20:29:53.108567 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.097473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45508 (local address 127.17.109.193:34663)
0806 20:29:53.100332 (+ 2859us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.100338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.100358 (+ 20us) server_negotiation.cc:407] Connection header received
0806 20:29:53.100428 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.100433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.100584 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.100778 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:53.101438 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.103055 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.105578 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.107015 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.107870 (+ 855us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.108028 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.108097 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.108255 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2722,"thread_start_us":2552,"threads_started":1}
I20260806 20:29:53.109304 3870 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd66459670742489e8420be2b06bd36 (DEFAULT_TABLE table=test-table [id=1e9d1d1b38904a31a4551a7755e3ef15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:53.110122 4026 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd66459670742489e8420be2b06bd36 (DEFAULT_TABLE table=test-table [id=1e9d1d1b38904a31a4551a7755e3ef15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:53.110924 3870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd66459670742489e8420be2b06bd36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:53.111387 4026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd66459670742489e8420be2b06bd36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:53.112116 3727 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd66459670742489e8420be2b06bd36 (DEFAULT_TABLE table=test-table [id=1e9d1d1b38904a31a4551a7755e3ef15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:53.113444 3727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd66459670742489e8420be2b06bd36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:53.123538 4118 tablet_bootstrap.cc:492] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Bootstrap starting.
I20260806 20:29:53.127388 4118 tablet_bootstrap.cc:654] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:53.128297 4119 tablet_bootstrap.cc:492] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Bootstrap starting.
I20260806 20:29:53.128775 4118 log.cc:824] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:53.130003 4120 tablet_bootstrap.cc:492] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Bootstrap starting.
I20260806 20:29:53.132205 4119 tablet_bootstrap.cc:654] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:53.133296 4120 tablet_bootstrap.cc:654] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:53.133448 4118 tablet_bootstrap.cc:492] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: No bootstrap required, opened a new log
I20260806 20:29:53.133505 4119 log.cc:824] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:53.133818 4118 ts_tablet_manager.cc:1397] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260806 20:29:53.134646 4120 log.cc:824] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:53.137168 4119 tablet_bootstrap.cc:492] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: No bootstrap required, opened a new log
I20260806 20:29:53.137435 4119 ts_tablet_manager.cc:1397] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260806 20:29:53.140851 4120 tablet_bootstrap.cc:492] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: No bootstrap required, opened a new log
I20260806 20:29:53.141122 4120 ts_tablet_manager.cc:1397] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260806 20:29:53.145040 4118 raft_consensus.cc:348] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.145584 4118 raft_consensus.cc:374] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:53.145695 4118 raft_consensus.cc:729] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d093434dc57431a8944bc154c0ab55d, State: Initialized, Role: FOLLOWER
I20260806 20:29:53.146370 4118 consensus_queue.cc:260] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.147622 4094 heartbeater.cc:503] Master 127.17.109.254:35941 was elected leader, sending a full tablet report...
I20260806 20:29:53.148298 4118 ts_tablet_manager.cc:1428] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Time spent starting tablet: real 0.014s user 0.001s sys 0.012s
I20260806 20:29:53.149462 4119 raft_consensus.cc:348] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.149842 4119 raft_consensus.cc:374] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:53.149969 4119 raft_consensus.cc:729] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb22fce8bf484312ab06296959095374, State: Initialized, Role: FOLLOWER
I20260806 20:29:53.150573 4119 consensus_queue.cc:260] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.150805 4120 raft_consensus.cc:348] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.151391 4120 raft_consensus.cc:374] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:53.151773 3946 heartbeater.cc:503] Master 127.17.109.254:35941 was elected leader, sending a full tablet report...
I20260806 20:29:53.151549 4120 raft_consensus.cc:729] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77d62cb793934616932c794c2ad9404f, State: Initialized, Role: FOLLOWER
I20260806 20:29:53.152412 4119 ts_tablet_manager.cc:1428] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260806 20:29:53.152922 4120 consensus_queue.cc:260] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.155241 4120 ts_tablet_manager.cc:1428] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
W20260806 20:29:53.213943 3947 tablet.cc:2369] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:29:53.260295 4095 tablet.cc:2369] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:29:53.319283 4126 raft_consensus.cc:482] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:29:53.319736 4126 raft_consensus.cc:504] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.321714 4126 leader_election.cc:288] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb22fce8bf484312ab06296959095374 (127.17.109.194:32929), 9d093434dc57431a8944bc154c0ab55d (127.17.109.195:39165)
I20260806 20:29:53.331691 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.322396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:35653 (local address 127.17.109.195:39165)
0806 20:29:53.322692 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.322707 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.323460 (+ 753us) server_negotiation.cc:407] Connection header received
0806 20:29:53.323666 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.323671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.323756 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.323870 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:53.324784 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.325940 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.329122 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.330396 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.330562 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.330628 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.331226 (+ 598us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.331365 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 20:29:53.331754 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.322330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:56631 (local address 127.17.109.194:32929)
0806 20:29:53.322586 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.322594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.323460 (+ 866us) server_negotiation.cc:407] Connection header received
0806 20:29:53.323665 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.323671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.323767 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.323873 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:53.324785 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.325940 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.329116 (+ 3176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.330372 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.330546 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.330621 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.331264 (+ 643us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.331405 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 20:29:53.331962 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.322271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:39165 (local address 127.17.109.193:35653)
0806 20:29:53.323270 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:53.323296 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:53.323501 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:53.323929 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:53.323936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:53.323988 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:53.324589 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.324609 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.326101 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.326107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.328939 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.328956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.330709 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.330722 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.330984 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.331055 (+ 71us) client_negotiation.cc:769] Sending connection context
0806 20:29:53.331189 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 20:29:53.331344 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":166,"spinlock_wait_cycles":63488,"thread_start_us":636,"threads_started":1}
I20260806 20:29:53.332610 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.322335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:32929 (local address 127.17.109.193:56631)
0806 20:29:53.323313 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:53.323334 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:53.323491 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:53.323929 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:53.323936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:53.323987 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:53.324616 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.324624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.326101 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.326107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.328939 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:53.328956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.330759 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.330765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:53.331053 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.331083 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 20:29:53.331980 (+ 897us) client_negotiation.cc:240] Negotiation successful
0806 20:29:53.332148 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"mutex_wait_us":305,"spinlock_wait_cycles":47872,"thread_start_us":467,"threads_started":1}
I20260806 20:29:53.333350 4048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd66459670742489e8420be2b06bd36" candidate_uuid: "77d62cb793934616932c794c2ad9404f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d093434dc57431a8944bc154c0ab55d" is_pre_election: true
I20260806 20:29:53.333815 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd66459670742489e8420be2b06bd36" candidate_uuid: "77d62cb793934616932c794c2ad9404f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb22fce8bf484312ab06296959095374" is_pre_election: true
I20260806 20:29:53.334008 4048 raft_consensus.cc:2454] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77d62cb793934616932c794c2ad9404f in term 0.
I20260806 20:29:53.334384 3891 raft_consensus.cc:2454] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 77d62cb793934616932c794c2ad9404f in term 0.
I20260806 20:29:53.334790 3679 leader_election.cc:302] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77d62cb793934616932c794c2ad9404f, 9d093434dc57431a8944bc154c0ab55d; no voters:
I20260806 20:29:53.335254 4126 raft_consensus.cc:2781] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:29:53.335412 4126 raft_consensus.cc:482] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:29:53.335539 4126 raft_consensus.cc:3037] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:53.337803 4126 raft_consensus.cc:504] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.338474 4126 leader_election.cc:288] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [CANDIDATE]: Term 1 election: Requested vote from peers bb22fce8bf484312ab06296959095374 (127.17.109.194:32929), 9d093434dc57431a8944bc154c0ab55d (127.17.109.195:39165)
I20260806 20:29:53.338989 3891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd66459670742489e8420be2b06bd36" candidate_uuid: "77d62cb793934616932c794c2ad9404f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb22fce8bf484312ab06296959095374"
I20260806 20:29:53.339059 4048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd66459670742489e8420be2b06bd36" candidate_uuid: "77d62cb793934616932c794c2ad9404f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d093434dc57431a8944bc154c0ab55d"
I20260806 20:29:53.339254 4048 raft_consensus.cc:3037] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:53.339243 3891 raft_consensus.cc:3037] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:53.342569 4048 raft_consensus.cc:2454] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77d62cb793934616932c794c2ad9404f in term 1.
I20260806 20:29:53.342573 3891 raft_consensus.cc:2454] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77d62cb793934616932c794c2ad9404f in term 1.
I20260806 20:29:53.343058 3680 leader_election.cc:302] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77d62cb793934616932c794c2ad9404f, bb22fce8bf484312ab06296959095374; no voters:
I20260806 20:29:53.343413 4126 raft_consensus.cc:2781] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:29:53.343997 4126 raft_consensus.cc:686] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 1 LEADER]: Becoming Leader. State: Replica: 77d62cb793934616932c794c2ad9404f, State: Running, Role: LEADER
I20260806 20:29:53.344604 4126 consensus_queue.cc:237] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } }
I20260806 20:29:53.350132 3594 catalog_manager.cc:5495] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f reported cstate change: term changed from 0 to 1, leader changed from <none> to 77d62cb793934616932c794c2ad9404f (127.17.109.193). New cstate: current_term: 1 leader_uuid: "77d62cb793934616932c794c2ad9404f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77d62cb793934616932c794c2ad9404f" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 34663 } health_report { overall_health: HEALTHY } } }
W20260806 20:29:53.381750 3797 tablet.cc:2369] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:29:53.402729 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.396995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.17.109.193:34663)
0806 20:29:53.397217 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.397222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.397243 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:29:53.397380 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.397383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.397471 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.397633 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:29:53.398002 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.398820 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.400419 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.400905 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.401005 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.401505 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:29:53.402128 (+ 623us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:29:53.402259 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.402393 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.402467 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 20:29:53.411656 4048 raft_consensus.cc:1261] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 1 FOLLOWER]: Refusing update from remote peer 77d62cb793934616932c794c2ad9404f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:29:53.411659 3891 raft_consensus.cc:1261] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 1 FOLLOWER]: Refusing update from remote peer 77d62cb793934616932c794c2ad9404f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:29:53.412695 4129 consensus_queue.cc:1035] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb22fce8bf484312ab06296959095374" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 32929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:29:53.413092 4126 consensus_queue.cc:1035] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d093434dc57431a8944bc154c0ab55d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 39165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:29:53.424810 4133 mvcc.cc:205] Tried to move back new op lower bound from 7315653400203399168 to 7315653399944495104. Current Snapshot: MvccSnapshot[applied={T|T < 7315653400203399168}]
I20260806 20:29:53.439726 3594 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:29:53.440008 3594 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:29:53.443598 3594 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a7637c6cfe8481490902622b7f3c63f: Sending DeleteTablet for 3 replicas of tablet 6cd66459670742489e8420be2b06bd36
I20260806 20:29:53.444941 3870 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd66459670742489e8420be2b06bd36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:48976
I20260806 20:29:53.445101 4026 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd66459670742489e8420be2b06bd36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:59384
I20260806 20:29:53.445148 3727 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd66459670742489e8420be2b06bd36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:53 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:45508
I20260806 20:29:53.447685 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 3659
I20260806 20:29:53.456136 4143 tablet_replica.cc:330] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: stopping tablet replica
I20260806 20:29:53.457381 4143 raft_consensus.cc:2229] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:29:53.458578 4143 raft_consensus.cc:2258] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:53.467067 4144 tablet_replica.cc:330] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: stopping tablet replica
I20260806 20:29:53.467932 4145 tablet_replica.cc:330] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: stopping tablet replica
I20260806 20:29:53.468513 4144 raft_consensus.cc:2229] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:29:53.469069 4145 raft_consensus.cc:2229] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:29:53.469982 4145 raft_consensus.cc:2258] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:53.470553 4144 raft_consensus.cc:2258] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:53.474148 4144 ts_tablet_manager.cc:1905] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:29:53.475752 4145 ts_tablet_manager.cc:1905] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 20:29:53 dist-test-slave-6k22 krb5kdc[3547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:29:53.477700 4143 ts_tablet_manager.cc:1905] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:29:53.486428 4145 ts_tablet_manager.cc:1918] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:29:53.486683 4145 log.cc:1196] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/6cd66459670742489e8420be2b06bd36
I20260806 20:29:53.487310 4145 ts_tablet_manager.cc:1939] T 6cd66459670742489e8420be2b06bd36 P 77d62cb793934616932c794c2ad9404f: Deleting consensus metadata
I20260806 20:29:53.488826 4144 ts_tablet_manager.cc:1918] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:29:53.489028 4144 log.cc:1196] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/6cd66459670742489e8420be2b06bd36
I20260806 20:29:53.489624 4144 ts_tablet_manager.cc:1939] T 6cd66459670742489e8420be2b06bd36 P 9d093434dc57431a8944bc154c0ab55d: Deleting consensus metadata
I20260806 20:29:53.489557 3578 catalog_manager.cc:4841] TS 77d62cb793934616932c794c2ad9404f (127.17.109.193:34663): tablet 6cd66459670742489e8420be2b06bd36 (table test-table [id=1e9d1d1b38904a31a4551a7755e3ef15]) successfully deleted
I20260806 20:29:53.491201 3580 catalog_manager.cc:4841] TS 9d093434dc57431a8944bc154c0ab55d (127.17.109.195:39165): tablet 6cd66459670742489e8420be2b06bd36 (table test-table [id=1e9d1d1b38904a31a4551a7755e3ef15]) successfully deleted
I20260806 20:29:53.491778 4143 ts_tablet_manager.cc:1918] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:29:53.492069 4143 log.cc:1196] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/6cd66459670742489e8420be2b06bd36
I20260806 20:29:53.492756 4143 ts_tablet_manager.cc:1939] T 6cd66459670742489e8420be2b06bd36 P bb22fce8bf484312ab06296959095374: Deleting consensus metadata
I20260806 20:29:53.494264 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.449770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45522 (local address 127.17.109.193:34663)
0806 20:29:53.450054 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.450063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.463056 (+ 12993us) server_negotiation.cc:407] Connection header received
0806 20:29:53.463491 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.463502 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.463669 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.463807 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:53.467706 (+ 3899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.468942 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.470945 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.471691 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.471818 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.477750 (+ 5932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:53.477764 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:53.477780 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:53.477836 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:53.482287 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.491140 (+ 8853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.491150 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.491161 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.491402 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.492049 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.492056 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.492061 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.492616 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:53.492926 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.493614 (+ 688us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.493900 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 20:29:53.495076 3581 catalog_manager.cc:4841] TS bb22fce8bf484312ab06296959095374 (127.17.109.194:32929): tablet 6cd66459670742489e8420be2b06bd36 (table test-table [id=1e9d1d1b38904a31a4551a7755e3ef15]) successfully deleted
I20260806 20:29:53.495992 3790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:53.600605 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 3659
I20260806 20:29:53.621883 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 3802
Aug 06 20:29:53 dist-test-slave-6k22 krb5kdc[3547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:29:53.637841 4113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.622820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48992 (local address 127.17.109.194:32929)
0806 20:29:53.623016 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.623020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.623041 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:29:53.623105 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.623109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.623181 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.623283 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:53.624662 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.625476 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.626733 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.627140 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.627244 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.631762 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:53.631790 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:53.631815 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:53.631882 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:53.635018 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.635857 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.635865 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.635867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.636022 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.636362 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.636365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.636367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.636786 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:53.637015 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.637256 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.637419 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:29:53.638561 3941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:53.731241 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 3802
I20260806 20:29:53.753358 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 3953
Aug 06 20:29:53 dist-test-slave-6k22 krb5kdc[3547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:29:53.769492 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:53.754129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59388 (local address 127.17.109.195:39165)
0806 20:29:53.754326 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:53.754331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:53.754349 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:29:53.754515 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:53.754519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:53.754584 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:53.754669 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:53.756059 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.756849 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.758180 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:53.758697 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:53.758820 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:53.763829 (+ 5009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:53.763845 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:53.763861 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:53.763908 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:53.766785 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.767439 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.767445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.767448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.767655 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:53.768089 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:53.768092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:53.768094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:53.768440 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:53.768658 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:53.768955 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 20:29:53.769135 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:29:53.770195 4089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:53.862424 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 3953
I20260806 20:29:53.883136 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 3560
I20260806 20:29:53.884548 3627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:53.997923 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 3560
2026-08-06T20:29:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4792 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T20:29:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:29:54Z Disabled control of system clock
I20260806 20:29:54.098651 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:41181
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:32867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:41181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 20:29:54.452592 4154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:54.452926 4154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:54.453011 4154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:54.460485 4154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:29:54.460601 4154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:54.460639 4154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:54.460700 4154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:29:54.460752 4154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:29:54.470505 4154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:32867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:41181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:41181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:54.471931 4154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:54.473771 4154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:54.486783 4160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:54.486742 4162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:54.487155 4159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:54.487694 4154 server_base.cc:1034] running on GCE node
I20260806 20:29:54.488631 4154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:54.490231 4154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:54.491469 4154 hybrid_clock.cc:648] HybridClock initialized: now 1786048194491418 us; error 46 us; skew 500 ppm
I20260806 20:29:54.491917 4154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:29:54.494422 4154 webserver.cc:460] Webserver started at http://127.17.109.254:36475/ using document root <none> and password file <none>
I20260806 20:29:54.495162 4154 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:54.495289 4154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:54.495687 4154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:54.498756 4154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "6b0d17f7756a4ff7b9739c148f925586"
format_stamp: "Formatted at 2026-08-06 20:29:54 on dist-test-slave-6k22"
server_key: "26635ab85e53e26f7905b9c3f2d4b629"
server_key_iv: "907807c6ab4cc6c848d5e0c046e6c6c8"
server_key_version: "encryptionkey@0"
I20260806 20:29:54.499661 4154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "6b0d17f7756a4ff7b9739c148f925586"
format_stamp: "Formatted at 2026-08-06 20:29:54 on dist-test-slave-6k22"
server_key: "26635ab85e53e26f7905b9c3f2d4b629"
server_key_iv: "907807c6ab4cc6c848d5e0c046e6c6c8"
server_key_version: "encryptionkey@0"
I20260806 20:29:54.505213 4154 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 20:29:54.508903 4169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:54.510401 4154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:29:54.510592 4154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "6b0d17f7756a4ff7b9739c148f925586"
format_stamp: "Formatted at 2026-08-06 20:29:54 on dist-test-slave-6k22"
server_key: "26635ab85e53e26f7905b9c3f2d4b629"
server_key_iv: "907807c6ab4cc6c848d5e0c046e6c6c8"
server_key_version: "encryptionkey@0"
I20260806 20:29:54.510829 4154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:54.541638 4154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:54.542695 4154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:54.543038 4154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:54.563385 4154 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:41181
I20260806 20:29:54.563405 4225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:41181 every 8 connection(s)
I20260806 20:29:54.564924 4154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:29:54.569017 4226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:54.570528 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 4154
I20260806 20:29:54.570746 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:29:54.571249 17847 external_mini_cluster.cc:1428] Setting key 0c4970927479c845532f93e9d8fe9c03
I20260806 20:29:54.582147 4226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586: Bootstrap starting.
I20260806 20:29:54.584901 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:54.573283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.17.109.254:41181)
0806 20:29:54.574398 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:54.574413 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:54.574458 (+ 45us) server_negotiation.cc:407] Connection header received
0806 20:29:54.576128 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:54.576150 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:54.576653 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:54.577055 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:54.577650 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:54.578944 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:54.580445 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:54.581106 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:54.581331 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:54.582572 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:54.582579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:54.582594 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:29:54.583140 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:54.583241 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:29:54.583266 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:54.583393 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:54.583639 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 20:29:54.583960 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":415,"threads_started":1}
I20260806 20:29:54.585872 4226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:54.587005 4226 log.cc:824] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:54.590193 4226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586: No bootstrap required, opened a new log
I20260806 20:29:54.597812 4226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0d17f7756a4ff7b9739c148f925586" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41181 } }
I20260806 20:29:54.598198 4226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:54.598296 4226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0d17f7756a4ff7b9739c148f925586, State: Initialized, Role: FOLLOWER
I20260806 20:29:54.598883 4226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0d17f7756a4ff7b9739c148f925586" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41181 } }
I20260806 20:29:54.599140 4226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:29:54.599298 4226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:29:54.599483 4226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:54.601711 4226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0d17f7756a4ff7b9739c148f925586" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41181 } }
I20260806 20:29:54.602221 4226 leader_election.cc:302] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b0d17f7756a4ff7b9739c148f925586; no voters:
I20260806 20:29:54.602862 4226 leader_election.cc:288] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:29:54.602999 4232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:29:54.604535 4232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [term 1 LEADER]: Becoming Leader. State: Replica: 6b0d17f7756a4ff7b9739c148f925586, State: Running, Role: LEADER
I20260806 20:29:54.605054 4232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0d17f7756a4ff7b9739c148f925586" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41181 } }
I20260806 20:29:54.605230 4226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:29:54.607963 4235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b0d17f7756a4ff7b9739c148f925586. Latest consensus state: current_term: 1 leader_uuid: "6b0d17f7756a4ff7b9739c148f925586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0d17f7756a4ff7b9739c148f925586" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41181 } } }
I20260806 20:29:54.608189 4233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b0d17f7756a4ff7b9739c148f925586" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b0d17f7756a4ff7b9739c148f925586" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 41181 } } }
I20260806 20:29:54.608457 4235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:54.608465 4233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586 [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:54.614087 4240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:29:54.619933 4240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:29:54.632845 4240 catalog_manager.cc:1347] Generated new cluster ID: a59ec3573bfe4b668c45e25daf99dc9e
I20260806 20:29:54.632987 4240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:29:54.657321 4240 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:29:54.658691 4240 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:29:54.667745 4240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586: Generated new TSK 0
I20260806 20:29:54.668604 4240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:29:54.684388 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41181
--builtin_ntp_servers=127.17.109.212:32867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 20:29:55.053354 4254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:55.053681 4254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:55.053889 4254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:55.062054 4254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:55.062175 4254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:55.062340 4254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:29:55.072353 4254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:32867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:55.073730 4254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:55.075672 4254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:55.090451 4262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:55.090418 4265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:55.090754 4263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:55.093292 4254 server_base.cc:1034] running on GCE node
I20260806 20:29:55.094012 4254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:55.095115 4254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:55.096386 4254 hybrid_clock.cc:648] HybridClock initialized: now 1786048195096326 us; error 49 us; skew 500 ppm
I20260806 20:29:55.096755 4254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:29:55.099202 4254 webserver.cc:460] Webserver started at http://127.17.109.193:46115/ using document root <none> and password file <none>
I20260806 20:29:55.100014 4254 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:55.100144 4254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:55.100474 4254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:55.103477 4254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "6e72442478b0483bb6d1d843f4e5c05e"
format_stamp: "Formatted at 2026-08-06 20:29:55 on dist-test-slave-6k22"
server_key: "8b6a6455a7a9708c72d87fffeff31d5c"
server_key_iv: "f552227857bebf10ed651633f0da1676"
server_key_version: "encryptionkey@0"
I20260806 20:29:55.104355 4254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "6e72442478b0483bb6d1d843f4e5c05e"
format_stamp: "Formatted at 2026-08-06 20:29:55 on dist-test-slave-6k22"
server_key: "8b6a6455a7a9708c72d87fffeff31d5c"
server_key_iv: "f552227857bebf10ed651633f0da1676"
server_key_version: "encryptionkey@0"
I20260806 20:29:55.109938 4254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:29:55.113480 4271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:55.114970 4254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:29:55.115108 4254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "6e72442478b0483bb6d1d843f4e5c05e"
format_stamp: "Formatted at 2026-08-06 20:29:55 on dist-test-slave-6k22"
server_key: "8b6a6455a7a9708c72d87fffeff31d5c"
server_key_iv: "f552227857bebf10ed651633f0da1676"
server_key_version: "encryptionkey@0"
I20260806 20:29:55.115306 4254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:55.138631 4254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:55.139496 4254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:55.139873 4254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:55.140933 4254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:55.142483 4254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:55.142572 4254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:55.142721 4254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:55.142814 4254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:55.178076 4254 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:36593
I20260806 20:29:55.178144 4396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:36593 every 8 connection(s)
I20260806 20:29:55.179409 4254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:29:55.182639 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 4254
I20260806 20:29:55.182849 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:29:55.183518 17847 external_mini_cluster.cc:1428] Setting key a1404e7f8d835aa658f255d5c5d93776
I20260806 20:29:55.186867 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41181
--builtin_ntp_servers=127.17.109.212:32867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 20:29:55.194589 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:55.182145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:48527 (local address 127.17.109.254:41181)
0806 20:29:55.182683 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:55.182689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:55.184136 (+ 1447us) server_negotiation.cc:407] Connection header received
0806 20:29:55.186583 (+ 2447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:55.186587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:55.186689 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:55.186789 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:55.188341 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:55.189066 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:55.191667 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:55.192077 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:55.192194 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:55.193835 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:55.193839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:55.193842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:29:55.193860 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:55.193899 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:29:55.193911 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:55.193976 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:55.194260 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 20:29:55.194337 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":255,"threads_started":1}
I20260806 20:29:55.196326 4403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:55.182746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:41181 (local address 127.17.109.193:48527)
0806 20:29:55.183954 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:55.184012 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:55.186274 (+ 2262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:55.187089 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:55.187105 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:55.187245 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:55.188156 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:55.188188 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:55.189246 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:55.189254 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:55.191491 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:55.191511 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:55.192312 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:55.192322 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:55.192677 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:55.193456 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:29:55.193461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:55.193522 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:29:55.193634 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:55.194004 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:55.194018 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:55.194029 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 20:29:55.194235 (+ 206us) client_negotiation.cc:240] Negotiation successful
0806 20:29:55.194678 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":345,"threads_started":1}
I20260806 20:29:55.198087 4397 heartbeater.cc:347] Connected to a master server at 127.17.109.254:41181
I20260806 20:29:55.198414 4397 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:55.199188 4397 heartbeater.cc:511] Master 127.17.109.254:41181 requested a full tablet report, sending...
I20260806 20:29:55.201362 4188 ts_manager.cc:194] Registered new tserver with Master: 6e72442478b0483bb6d1d843f4e5c05e (127.17.109.193:36593)
I20260806 20:29:55.202839 4188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.193:48527
W20260806 20:29:55.529798 4404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:55.530135 4404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:55.530300 4404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:55.537745 4404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:55.537918 4404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:55.538123 4404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:29:55.548316 4404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:32867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:55.549773 4404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:55.552095 4404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:55.565660 4411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:55.565694 4413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:55.566617 4409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:55.567617 4404 server_base.cc:1034] running on GCE node
I20260806 20:29:55.568399 4404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:55.569407 4404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:55.570640 4404 hybrid_clock.cc:648] HybridClock initialized: now 1786048195570580 us; error 54 us; skew 500 ppm
I20260806 20:29:55.570978 4404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:29:55.573413 4404 webserver.cc:460] Webserver started at http://127.17.109.194:43955/ using document root <none> and password file <none>
I20260806 20:29:55.574127 4404 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:55.574255 4404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:55.574558 4404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:55.577586 4404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "7e91f18c54ca4aa28b0aa445252f06d7"
format_stamp: "Formatted at 2026-08-06 20:29:55 on dist-test-slave-6k22"
server_key: "ca32d0eda8eaa7239910063855bfa5da"
server_key_iv: "c611953e9bff6e0cab19b668e9458ae7"
server_key_version: "encryptionkey@0"
I20260806 20:29:55.578418 4404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "7e91f18c54ca4aa28b0aa445252f06d7"
format_stamp: "Formatted at 2026-08-06 20:29:55 on dist-test-slave-6k22"
server_key: "ca32d0eda8eaa7239910063855bfa5da"
server_key_iv: "c611953e9bff6e0cab19b668e9458ae7"
server_key_version: "encryptionkey@0"
I20260806 20:29:55.583734 4404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260806 20:29:55.587358 4420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:55.589095 4404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:55.589322 4404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "7e91f18c54ca4aa28b0aa445252f06d7"
format_stamp: "Formatted at 2026-08-06 20:29:55 on dist-test-slave-6k22"
server_key: "ca32d0eda8eaa7239910063855bfa5da"
server_key_iv: "c611953e9bff6e0cab19b668e9458ae7"
server_key_version: "encryptionkey@0"
I20260806 20:29:55.589581 4404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:55.604427 4404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:55.605406 4404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:55.605752 4404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:55.606930 4404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:55.608556 4404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:55.608703 4404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:55.608842 4404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:55.608938 4404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:55.656056 4404 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:40829
I20260806 20:29:55.656131 4543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:40829 every 8 connection(s)
I20260806 20:29:55.658183 4404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:29:55.662858 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 4404
I20260806 20:29:55.663040 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:29:55.663605 17847 external_mini_cluster.cc:1428] Setting key e018fac782c08d09b33a2c127f958ff0
I20260806 20:29:55.666854 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41181
--builtin_ntp_servers=127.17.109.212:32867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 20:29:55.677480 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:55.661394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:48181 (local address 127.17.109.254:41181)
0806 20:29:55.661636 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:55.661646 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:55.663478 (+ 1832us) server_negotiation.cc:407] Connection header received
0806 20:29:55.666525 (+ 3047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:55.666534 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:55.666628 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:55.666745 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:55.668444 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:55.670217 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:55.673360 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:55.674046 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:55.674200 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:55.676323 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:55.676332 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:55.676339 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:29:55.676372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:55.676428 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:29:55.676452 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:55.676573 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:55.676880 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 20:29:55.677014 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 20:29:55.679075 4546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:55.661894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:41181 (local address 127.17.109.194:48181)
0806 20:29:55.663252 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:55.663319 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:55.666160 (+ 2841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:55.667042 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:55.667058 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:55.667210 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:55.668232 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:55.668265 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:55.670374 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:55.670382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:55.673189 (+ 2807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:55.673209 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:55.674494 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:55.674504 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:55.674839 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:55.675798 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:29:55.675808 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:55.675866 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:29:55.675975 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:55.676569 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:55.676581 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:55.676592 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 20:29:55.676832 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 20:29:55.677286 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":388,"threads_started":1}
I20260806 20:29:55.680835 4544 heartbeater.cc:347] Connected to a master server at 127.17.109.254:41181
I20260806 20:29:55.681149 4544 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:55.681984 4544 heartbeater.cc:511] Master 127.17.109.254:41181 requested a full tablet report, sending...
I20260806 20:29:55.683787 4188 ts_manager.cc:194] Registered new tserver with Master: 7e91f18c54ca4aa28b0aa445252f06d7 (127.17.109.194:40829)
I20260806 20:29:55.685168 4188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.194:48181
W20260806 20:29:56.017413 4547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:56.017709 4547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:56.017797 4547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:56.025306 4547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:56.025417 4547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:56.025525 4547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:29:56.034976 4547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:32867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:41181
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:56.036408 4547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:56.038182 4547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:56.052420 4552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:56.052464 4556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:56.052501 4553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:56.053669 4547 server_base.cc:1034] running on GCE node
I20260806 20:29:56.054296 4547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:56.055296 4547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:56.056519 4547 hybrid_clock.cc:648] HybridClock initialized: now 1786048196056454 us; error 53 us; skew 500 ppm
I20260806 20:29:56.056836 4547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:29:56.059178 4547 webserver.cc:460] Webserver started at http://127.17.109.195:33269/ using document root <none> and password file <none>
I20260806 20:29:56.059978 4547 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:56.060122 4547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:56.060532 4547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:56.063762 4547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "f8c22999c0b140b484d6345f7613fcdf"
format_stamp: "Formatted at 2026-08-06 20:29:56 on dist-test-slave-6k22"
server_key: "dbe1077d244ddeb3b588f679a02aa1b5"
server_key_iv: "ad32edc9771c45a8f55963f207e1dda2"
server_key_version: "encryptionkey@0"
I20260806 20:29:56.064592 4547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "f8c22999c0b140b484d6345f7613fcdf"
format_stamp: "Formatted at 2026-08-06 20:29:56 on dist-test-slave-6k22"
server_key: "dbe1077d244ddeb3b588f679a02aa1b5"
server_key_iv: "ad32edc9771c45a8f55963f207e1dda2"
server_key_version: "encryptionkey@0"
I20260806 20:29:56.070273 4547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:29:56.074105 4563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:56.075645 4547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:56.075834 4547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "f8c22999c0b140b484d6345f7613fcdf"
format_stamp: "Formatted at 2026-08-06 20:29:56 on dist-test-slave-6k22"
server_key: "dbe1077d244ddeb3b588f679a02aa1b5"
server_key_iv: "ad32edc9771c45a8f55963f207e1dda2"
server_key_version: "encryptionkey@0"
I20260806 20:29:56.076059 4547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:56.095527 4547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:56.096457 4547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:56.096774 4547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:56.097783 4547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:56.099337 4547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:56.099459 4547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:56.099599 4547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:56.099690 4547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260806 20:29:56.134724 4547 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:33977
I20260806 20:29:56.134830 4684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:33977 every 8 connection(s)
I20260806 20:29:56.136119 4547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:29:56.142534 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 4547
I20260806 20:29:56.142765 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:29:56.143203 17847 external_mini_cluster.cc:1428] Setting key f1cb2d570e67f4999fa2dc538a008b9f
I20260806 20:29:56.155230 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.138792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:49975 (local address 127.17.109.254:41181)
0806 20:29:56.138974 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.138978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.140338 (+ 1360us) server_negotiation.cc:407] Connection header received
0806 20:29:56.142141 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.142148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.142256 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.142366 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:56.143492 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.144550 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.146041 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.146444 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.146565 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.154310 (+ 7745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:56.154315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:56.154319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:29:56.154338 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:56.154373 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:29:56.154385 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:56.154480 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.154761 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.154878 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:29:56.156363 4688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.139277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:41181 (local address 127.17.109.195:49975)
0806 20:29:56.140185 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.140220 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:56.141883 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:56.142581 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:56.142589 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:56.142704 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:56.143331 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.143357 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.144693 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.144698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.145900 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.145911 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.146662 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.146669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.146901 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.153921 (+ 7020us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:29:56.153924 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:56.153969 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:29:56.154064 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:56.154495 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:56.154509 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:56.154521 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 20:29:56.154711 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 20:29:56.155178 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":311,"threads_started":1}
I20260806 20:29:56.157760 4685 heartbeater.cc:347] Connected to a master server at 127.17.109.254:41181
I20260806 20:29:56.158074 4685 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:56.158867 4685 heartbeater.cc:511] Master 127.17.109.254:41181 requested a full tablet report, sending...
I20260806 20:29:56.160331 4187 ts_manager.cc:194] Registered new tserver with Master: f8c22999c0b140b484d6345f7613fcdf (127.17.109.195:33977)
I20260806 20:29:56.161253 4187 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.195:49975
I20260806 20:29:56.171092 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:29:56.184932 4402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.177871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.17.109.254:41181)
0806 20:29:56.178088 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.178095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.178428 (+ 333us) server_negotiation.cc:407] Connection header received
0806 20:29:56.178667 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.178673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.178755 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.178849 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:56.179849 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.180943 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.182502 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.182942 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.183082 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.184201 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:56.184207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:56.184212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:29:56.184237 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:56.184272 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:29:56.184284 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:56.184375 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.184513 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.184608 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 20:29:56.189399 4187 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:29:56.197547 4187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:29:56.205778 4397 heartbeater.cc:503] Master 127.17.109.254:41181 was elected leader, sending a full tablet report...
I20260806 20:29:56.222404 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.210946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.17.109.194:40829)
0806 20:29:56.213052 (+ 2106us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.213063 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.213094 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:29:56.213208 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.213215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.213470 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.214436 (+ 966us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:56.214680 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.216516 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.219465 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.221014 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.221177 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.221409 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.221832 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.222017 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1860,"thread_start_us":1699,"threads_started":1}
I20260806 20:29:56.222638 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.210488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:40829 (local address 127.0.0.1:41766)
0806 20:29:56.211448 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.211495 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:56.211710 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:56.213746 (+ 2036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:56.213755 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:56.213786 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:56.214189 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.214204 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.216683 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.216695 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.219185 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.219197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.221243 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.221253 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.221523 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.221598 (+ 75us) client_negotiation.cc:769] Sending connection context
0806 20:29:56.221776 (+ 178us) client_negotiation.cc:240] Negotiation successful
0806 20:29:56.222002 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":1,"thread_start_us":529,"threads_started":1}
I20260806 20:29:56.223634 4699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.210828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36593 (local address 127.0.0.1:45184)
0806 20:29:56.212317 (+ 1489us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.212341 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:56.212443 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:56.214240 (+ 1797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:56.214247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:56.214277 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:56.215492 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.215507 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.217785 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.217794 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.220421 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.220446 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.222793 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.222799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.222961 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.222994 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 20:29:56.223075 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 20:29:56.223216 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1364,"mutex_wait_us":22,"thread_start_us":1211,"threads_started":1}
I20260806 20:29:56.224848 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.211434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45184 (local address 127.17.109.193:36593)
0806 20:29:56.213440 (+ 2006us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.213454 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.213493 (+ 39us) server_negotiation.cc:407] Connection header received
0806 20:29:56.213644 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.213653 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.213952 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.214305 (+ 353us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:56.215729 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.217625 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.220667 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.222222 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.222456 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.222752 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.224078 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.224252 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1776,"thread_start_us":1609,"threads_started":1}
I20260806 20:29:56.224937 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.212217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33977 (local address 127.0.0.1:59074)
0806 20:29:56.215087 (+ 2870us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.215110 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:56.215749 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:56.216550 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:56.216556 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:56.216588 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:56.217114 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.217140 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.219185 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.219190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.221658 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.221670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.224160 (+ 2490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.224167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.224333 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.224373 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 20:29:56.224467 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 20:29:56.224589 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2726,"mutex_wait_us":3,"thread_start_us":2561,"threads_started":1}
I20260806 20:29:56.225440 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.213006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.17.109.195:33977)
0806 20:29:56.213843 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.213854 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.215307 (+ 1453us) server_negotiation.cc:407] Connection header received
0806 20:29:56.215921 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.215929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.216304 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.216518 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:56.217362 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.219001 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.221916 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.223596 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.223808 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.224032 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.224762 (+ 730us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.224958 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":493,"threads_started":1}
I20260806 20:29:56.227528 4470 tablet_service.cc:1467] Processing CreateTablet for tablet 28ac085dd70e4a3099fdd9c20816122b (DEFAULT_TABLE table=test-table [id=e8f707bcffdd45b2906c75ae0b5b8d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:56.228649 4613 tablet_service.cc:1467] Processing CreateTablet for tablet 28ac085dd70e4a3099fdd9c20816122b (DEFAULT_TABLE table=test-table [id=e8f707bcffdd45b2906c75ae0b5b8d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:56.229270 4470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ac085dd70e4a3099fdd9c20816122b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:56.228766 4323 tablet_service.cc:1467] Processing CreateTablet for tablet 28ac085dd70e4a3099fdd9c20816122b (DEFAULT_TABLE table=test-table [id=e8f707bcffdd45b2906c75ae0b5b8d63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:29:56.229847 4613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ac085dd70e4a3099fdd9c20816122b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:56.230363 4323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28ac085dd70e4a3099fdd9c20816122b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:56.241050 4705 tablet_bootstrap.cc:492] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Bootstrap starting.
I20260806 20:29:56.244405 4705 tablet_bootstrap.cc:654] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:56.246181 4705 log.cc:824] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:56.248574 4706 tablet_bootstrap.cc:492] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Bootstrap starting.
I20260806 20:29:56.249316 4705 tablet_bootstrap.cc:492] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: No bootstrap required, opened a new log
I20260806 20:29:56.249378 4708 tablet_bootstrap.cc:492] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: Bootstrap starting.
I20260806 20:29:56.249598 4705 ts_tablet_manager.cc:1397] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260806 20:29:56.252600 4706 tablet_bootstrap.cc:654] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:56.252717 4708 tablet_bootstrap.cc:654] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:56.253892 4706 log.cc:824] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:56.254014 4708 log.cc:824] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:29:56.258157 4708 tablet_bootstrap.cc:492] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: No bootstrap required, opened a new log
I20260806 20:29:56.258545 4708 ts_tablet_manager.cc:1397] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260806 20:29:56.261183 4706 tablet_bootstrap.cc:492] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: No bootstrap required, opened a new log
I20260806 20:29:56.261466 4706 ts_tablet_manager.cc:1397] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260806 20:29:56.261861 4705 raft_consensus.cc:348] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.262341 4705 raft_consensus.cc:374] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:56.262457 4705 raft_consensus.cc:729] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8c22999c0b140b484d6345f7613fcdf, State: Initialized, Role: FOLLOWER
I20260806 20:29:56.263185 4705 consensus_queue.cc:260] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.264479 4685 heartbeater.cc:503] Master 127.17.109.254:41181 was elected leader, sending a full tablet report...
I20260806 20:29:56.265027 4705 ts_tablet_manager.cc:1428] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Time spent starting tablet: real 0.015s user 0.009s sys 0.006s
I20260806 20:29:56.269447 4708 raft_consensus.cc:348] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.270076 4708 raft_consensus.cc:374] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:56.270246 4708 raft_consensus.cc:729] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e72442478b0483bb6d1d843f4e5c05e, State: Initialized, Role: FOLLOWER
I20260806 20:29:56.270686 4706 raft_consensus.cc:348] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.271148 4706 raft_consensus.cc:374] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:56.270944 4708 consensus_queue.cc:260] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.271261 4706 raft_consensus.cc:729] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e91f18c54ca4aa28b0aa445252f06d7, State: Initialized, Role: FOLLOWER
I20260806 20:29:56.271943 4706 consensus_queue.cc:260] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.273172 4544 heartbeater.cc:503] Master 127.17.109.254:41181 was elected leader, sending a full tablet report...
I20260806 20:29:56.273681 4706 ts_tablet_manager.cc:1428] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260806 20:29:56.274058 4708 ts_tablet_manager.cc:1428] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: Time spent starting tablet: real 0.015s user 0.013s sys 0.004s
W20260806 20:29:56.387976 4686 tablet.cc:2369] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:29:56.410705 4545 tablet.cc:2369] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:29:56.432013 4400 tablet.cc:2369] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:29:56.461885 4713 raft_consensus.cc:482] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:29:56.462255 4713 raft_consensus.cc:504] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.464215 4713 leader_election.cc:288] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e91f18c54ca4aa28b0aa445252f06d7 (127.17.109.194:40829), f8c22999c0b140b484d6345f7613fcdf (127.17.109.195:33977)
I20260806 20:29:56.471372 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.464749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:44837 (local address 127.17.109.194:40829)
0806 20:29:56.464956 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.464960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.465377 (+ 417us) server_negotiation.cc:407] Connection header received
0806 20:29:56.465615 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.465685 (+ 70us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.465780 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.465928 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:56.466551 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.467301 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.469361 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.470353 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.470535 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.470631 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.470906 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.471044 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 20:29:56.471506 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.465689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:57817 (local address 127.17.109.195:33977)
0806 20:29:56.465923 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.465930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.466188 (+ 258us) server_negotiation.cc:407] Connection header received
0806 20:29:56.466416 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.466420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.466477 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.466571 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:29:56.467077 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.467853 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.469664 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.470583 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.470776 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.470844 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.471123 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.471264 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 20:29:56.471679 4717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.465383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33977 (local address 127.17.109.193:57817)
0806 20:29:56.466075 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.466094 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:56.466264 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:56.466589 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:56.466593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:56.466616 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:56.466929 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.466937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.468006 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.468010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.469486 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.469499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.470785 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.470791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.470959 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.470981 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 20:29:56.471083 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 20:29:56.471251 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":33,"thread_start_us":550,"threads_started":1}
I20260806 20:29:56.472127 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.464529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:40829 (local address 127.17.109.193:44837)
0806 20:29:56.465257 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.465278 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:56.465425 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:56.465959 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:56.465964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:56.465997 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:29:56.466394 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.466404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.467451 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.467457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.469144 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:56.469159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.470547 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.470553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:56.470732 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.470762 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 20:29:56.470878 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 20:29:56.471024 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":3,"thread_start_us":364,"threads_started":1}
I20260806 20:29:56.472954 4634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ac085dd70e4a3099fdd9c20816122b" candidate_uuid: "6e72442478b0483bb6d1d843f4e5c05e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8c22999c0b140b484d6345f7613fcdf" is_pre_election: true
I20260806 20:29:56.473273 4490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ac085dd70e4a3099fdd9c20816122b" candidate_uuid: "6e72442478b0483bb6d1d843f4e5c05e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" is_pre_election: true
I20260806 20:29:56.473652 4634 raft_consensus.cc:2454] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e72442478b0483bb6d1d843f4e5c05e in term 0.
I20260806 20:29:56.473881 4490 raft_consensus.cc:2454] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e72442478b0483bb6d1d843f4e5c05e in term 0.
I20260806 20:29:56.474488 4273 leader_election.cc:302] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e72442478b0483bb6d1d843f4e5c05e, f8c22999c0b140b484d6345f7613fcdf; no voters:
I20260806 20:29:56.474923 4713 raft_consensus.cc:2781] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:29:56.475093 4713 raft_consensus.cc:482] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:29:56.475246 4713 raft_consensus.cc:3037] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:56.477598 4713 raft_consensus.cc:504] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.478268 4713 leader_election.cc:288] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [CANDIDATE]: Term 1 election: Requested vote from peers 7e91f18c54ca4aa28b0aa445252f06d7 (127.17.109.194:40829), f8c22999c0b140b484d6345f7613fcdf (127.17.109.195:33977)
I20260806 20:29:56.478869 4634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ac085dd70e4a3099fdd9c20816122b" candidate_uuid: "6e72442478b0483bb6d1d843f4e5c05e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8c22999c0b140b484d6345f7613fcdf"
I20260806 20:29:56.478900 4490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28ac085dd70e4a3099fdd9c20816122b" candidate_uuid: "6e72442478b0483bb6d1d843f4e5c05e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e91f18c54ca4aa28b0aa445252f06d7"
I20260806 20:29:56.479079 4634 raft_consensus.cc:3037] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:56.479140 4490 raft_consensus.cc:3037] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:56.481487 4490 raft_consensus.cc:2454] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e72442478b0483bb6d1d843f4e5c05e in term 1.
I20260806 20:29:56.481724 4634 raft_consensus.cc:2454] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e72442478b0483bb6d1d843f4e5c05e in term 1.
I20260806 20:29:56.481927 4273 leader_election.cc:302] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e72442478b0483bb6d1d843f4e5c05e, 7e91f18c54ca4aa28b0aa445252f06d7; no voters:
I20260806 20:29:56.482249 4713 raft_consensus.cc:2781] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:29:56.482728 4713 raft_consensus.cc:686] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 1 LEADER]: Becoming Leader. State: Replica: 6e72442478b0483bb6d1d843f4e5c05e, State: Running, Role: LEADER
I20260806 20:29:56.483207 4713 consensus_queue.cc:237] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } }
I20260806 20:29:56.488164 4188 catalog_manager.cc:5495] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e72442478b0483bb6d1d843f4e5c05e (127.17.109.193). New cstate: current_term: 1 leader_uuid: "6e72442478b0483bb6d1d843f4e5c05e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e72442478b0483bb6d1d843f4e5c05e" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 36593 } health_report { overall_health: HEALTHY } } }
I20260806 20:29:56.524425 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.518759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45196 (local address 127.17.109.193:36593)
0806 20:29:56.518959 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.518964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.519082 (+ 118us) server_negotiation.cc:407] Connection header received
0806 20:29:56.519272 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.519276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.519374 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.519517 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:29:56.520020 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.520970 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.522386 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.522826 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.522926 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.523364 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:29:56.523935 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:29:56.524056 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.524157 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.524227 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:29:56.533143 4490 raft_consensus.cc:1261] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 1 FOLLOWER]: Refusing update from remote peer 6e72442478b0483bb6d1d843f4e5c05e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:29:56.533141 4634 raft_consensus.cc:1261] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 1 FOLLOWER]: Refusing update from remote peer 6e72442478b0483bb6d1d843f4e5c05e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:29:56.534197 4718 consensus_queue.cc:1035] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e91f18c54ca4aa28b0aa445252f06d7" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 40829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:29:56.534601 4713 consensus_queue.cc:1035] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8c22999c0b140b484d6345f7613fcdf" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:29:56.546747 4721 mvcc.cc:205] Tried to move back new op lower bound from 7315653412987502592 to 7315653412799090688. Current Snapshot: MvccSnapshot[applied={T|T < 7315653412987502592}]
I20260806 20:29:56.563010 4187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:29:56.563344 4187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:29:56.567740 4187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b0d17f7756a4ff7b9739c148f925586: Sending DeleteTablet for 3 replicas of tablet 28ac085dd70e4a3099fdd9c20816122b
I20260806 20:29:56.569201 4613 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ac085dd70e4a3099fdd9c20816122b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:56 UTC) from {username='slave'} at 127.0.0.1:59074
I20260806 20:29:56.569212 4323 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ac085dd70e4a3099fdd9c20816122b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:56 UTC) from {username='slave'} at 127.0.0.1:45184
I20260806 20:29:56.569208 4470 tablet_service.cc:1514] Processing DeleteTablet for tablet 28ac085dd70e4a3099fdd9c20816122b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:29:56 UTC) from {username='slave'} at 127.0.0.1:41766
I20260806 20:29:56.573165 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 4254
I20260806 20:29:56.589926 4732 tablet_replica.cc:330] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: stopping tablet replica
I20260806 20:29:56.590430 4732 raft_consensus.cc:2229] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:29:56.590997 4732 raft_consensus.cc:2258] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:56.591935 4733 tablet_replica.cc:330] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: stopping tablet replica
I20260806 20:29:56.592412 4733 raft_consensus.cc:2229] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:29:56.593255 4733 raft_consensus.cc:2258] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:56.595081 4731 tablet_replica.cc:330] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: stopping tablet replica
I20260806 20:29:56.595528 4731 raft_consensus.cc:2229] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:29:56.596083 4731 raft_consensus.cc:2258] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:29:56.596158 4732 ts_tablet_manager.cc:1905] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:29:56.598347 4731 ts_tablet_manager.cc:1905] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:29:56.602640 4733 ts_tablet_manager.cc:1905] T 28ac085dd70e4a3099fdd9c20816122b P 6e72442478b0483bb6d1d843f4e5c05e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:29:56.604151 4732 ts_tablet_manager.cc:1918] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:29:56.604337 4732 log.cc:1196] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/28ac085dd70e4a3099fdd9c20816122b
I20260806 20:29:56.604979 4732 ts_tablet_manager.cc:1939] T 28ac085dd70e4a3099fdd9c20816122b P 7e91f18c54ca4aa28b0aa445252f06d7: Deleting consensus metadata
I20260806 20:29:56.605728 4731 ts_tablet_manager.cc:1918] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:29:56.605907 4731 log.cc:1196] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/28ac085dd70e4a3099fdd9c20816122b
I20260806 20:29:56.606504 4731 ts_tablet_manager.cc:1939] T 28ac085dd70e4a3099fdd9c20816122b P f8c22999c0b140b484d6345f7613fcdf: Deleting consensus metadata
I20260806 20:29:56.607138 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.575676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45198 (local address 127.17.109.193:36593)
0806 20:29:56.576104 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.576114 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.594075 (+ 17961us) server_negotiation.cc:407] Connection header received
0806 20:29:56.594646 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.594652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.594775 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.594859 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:56.595666 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.596849 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.598952 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.599628 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.599740 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.605243 (+ 5503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:56.605254 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:56.605271 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:29:56.606078 (+ 807us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:56.606177 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:29:56.606207 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:56.606303 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.606618 (+ 315us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.606748 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307}
I20260806 20:29:56.607167 4171 catalog_manager.cc:4841] TS 7e91f18c54ca4aa28b0aa445252f06d7 (127.17.109.194:40829): tablet 28ac085dd70e4a3099fdd9c20816122b (table test-table [id=e8f707bcffdd45b2906c75ae0b5b8d63]) successfully deleted
I20260806 20:29:56.608268 4171 catalog_manager.cc:4841] TS f8c22999c0b140b484d6345f7613fcdf (127.17.109.195:33977): tablet 28ac085dd70e4a3099fdd9c20816122b (table test-table [id=e8f707bcffdd45b2906c75ae0b5b8d63]) successfully deleted
I20260806 20:29:56.608332 4391 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:56.707293 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 4254
I20260806 20:29:56.727981 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 4404
W20260806 20:29:56.727914 4173 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.109.193:36593 (error 108)
I20260806 20:29:56.730360 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.729542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36593 (local address 127.0.0.1:45204)
0806 20:29:56.730063 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.730199 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":268,"threads_started":1}
W20260806 20:29:56.730720 4173 catalog_manager.cc:4584] TS 6e72442478b0483bb6d1d843f4e5c05e (127.17.109.193:36593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28ac085dd70e4a3099fdd9c20816122b: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36593: connect: Connection refused (error 111)
I20260806 20:29:56.735735 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.728911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41776 (local address 127.17.109.194:40829)
0806 20:29:56.729087 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.729094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.729116 (+ 22us) server_negotiation.cc:407] Connection header received
0806 20:29:56.729345 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.729351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.729475 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.729586 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:56.730168 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.731225 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.732683 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.733087 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.733273 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.734296 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:56.734306 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:56.734319 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:29:56.734900 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:56.734953 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:29:56.734971 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:56.735062 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.735240 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.735327 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:29:56.736383 4539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:56.784381 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.783904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36593 (local address 127.0.0.1:45212)
0806 20:29:56.784119 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.784234 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 20:29:56.785465 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.785116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36593 (local address 127.0.0.1:45222)
0806 20:29:56.785273 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.785340 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 20:29:56.827180 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 4404
I20260806 20:29:56.846346 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.845878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36593 (local address 127.0.0.1:45236)
0806 20:29:56.846085 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.846189 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 20:29:56.847244 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 4547
I20260806 20:29:56.847669 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.847140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36593 (local address 127.0.0.1:45242)
0806 20:29:56.847353 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.847457 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 20:29:56.854143 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.848076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59078 (local address 127.17.109.195:33977)
0806 20:29:56.848241 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:56.848246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:56.848264 (+ 18us) server_negotiation.cc:407] Connection header received
0806 20:29:56.848347 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:56.848350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:56.848433 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:56.848518 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:56.849000 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.849692 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.851110 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:56.851481 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:56.851636 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:56.852876 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:56.852884 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:56.852900 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:29:56.853482 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:56.853530 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:29:56.853550 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:56.853639 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:56.853796 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 20:29:56.853886 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 20:29:56.854888 4680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:56.928812 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.928229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36593 (local address 127.0.0.1:45250)
0806 20:29:56.928482 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.928587 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260806 20:29:56.930094 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:56.929577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:36593 (local address 127.0.0.1:45262)
0806 20:29:56.929856 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:56.929941 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260806 20:29:56.945323 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 4547
I20260806 20:29:56.966014 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 4154
I20260806 20:29:56.967180 4220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:29:57.079769 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 4154
2026-08-06T20:29:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3077 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:29:57 dist-test-slave-6k22 krb5kdc[4744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:29:57 dist-test-slave-6k22 krb5kdc[4744](info): set up 2 sockets
Aug 06 20:29:57 dist-test-slave-6k22 krb5kdc[4744](info): commencing operation
krb5kdc: starting...
W20260806 20:29:58.307729 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:29:58 dist-test-slave-6k22 krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:29:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:29:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:29:58.645102 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:40013
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:59.006901 4757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:59.007229 4757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:59.007328 4757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:59.014786 4757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:29:59.014914 4757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:59.014969 4757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:59.015033 4757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:29:59.015079 4757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:29:59.025286 4757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:40013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:59.026605 4757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:59.028016 4757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:59.040591 4763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:59.040771 4764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:59.041361 4766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:59.041430 4757 server_base.cc:1034] running on GCE node
I20260806 20:29:59.042290 4757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:59.043812 4757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:59.045043 4757 hybrid_clock.cc:648] HybridClock initialized: now 1786048199044979 us; error 53 us; skew 500 ppm
I20260806 20:29:59.045423 4757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:59 dist-test-slave-6k22 krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048199, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:59.048776 4757 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:29:59.050247 4757 webserver.cc:460] Webserver started at http://127.17.109.254:44161/ using document root <none> and password file <none>
I20260806 20:29:59.050905 4757 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:59.051004 4757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:59.051265 4757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:59.054328 4757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "35f35cc07a264e57ab6b38799faefde1"
format_stamp: "Formatted at 2026-08-06 20:29:59 on dist-test-slave-6k22"
server_key: "8d491fd5dd96f69773823adc0d45b02f"
server_key_iv: "7b97f88444d959b3b196b2f8ca785a41"
server_key_version: "encryptionkey@0"
I20260806 20:29:59.055024 4757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "35f35cc07a264e57ab6b38799faefde1"
format_stamp: "Formatted at 2026-08-06 20:29:59 on dist-test-slave-6k22"
server_key: "8d491fd5dd96f69773823adc0d45b02f"
server_key_iv: "7b97f88444d959b3b196b2f8ca785a41"
server_key_version: "encryptionkey@0"
I20260806 20:29:59.060747 4757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 20:29:59.064927 4774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:59.066442 4757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260806 20:29:59.066633 4757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "35f35cc07a264e57ab6b38799faefde1"
format_stamp: "Formatted at 2026-08-06 20:29:59 on dist-test-slave-6k22"
server_key: "8d491fd5dd96f69773823adc0d45b02f"
server_key_iv: "7b97f88444d959b3b196b2f8ca785a41"
server_key_version: "encryptionkey@0"
I20260806 20:29:59.066844 4757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:59.093868 4757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:59.098237 4757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:59.098562 4757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:59.118361 4757 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:40013
I20260806 20:29:59.118422 4832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:40013 every 8 connection(s)
I20260806 20:29:59.119839 4757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:29:59.124352 4833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:29:59.128391 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 4757
I20260806 20:29:59.128589 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:29:59.129184 17847 external_mini_cluster.cc:1428] Setting key a76335fff7bcdcbd59a810f6276f9a05
I20260806 20:29:59.138729 4833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1: Bootstrap starting.
I20260806 20:29:59.142921 4833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1: Neither blocks nor log segments found. Creating new log.
I20260806 20:29:59.144457 4833 log.cc:824] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1: Log is configured to *not* fsync() on all Append() calls
Aug 06 20:29:59 dist-test-slave-6k22 krb5kdc[4744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:59.147986 4833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1: No bootstrap required, opened a new log
I20260806 20:29:59.158016 4836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:59.131308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.17.109.254:40013)
0806 20:29:59.132285 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:59.132300 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:59.132352 (+ 52us) server_negotiation.cc:407] Connection header received
0806 20:29:59.134161 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:59.134183 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:59.134606 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:59.134961 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:59.137052 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:59.138579 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:59.140628 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:59.141454 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:59.141618 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:59.147353 (+ 5735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:59.147366 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:59.147386 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:59.147442 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:59.152437 (+ 4995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:59.153267 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:59.153276 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:59.153284 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:59.153534 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:59.154033 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:59.154039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:59.154043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:59.154842 (+ 799us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:59.155194 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:59.155745 (+ 551us) server_negotiation.cc:299] Negotiation successful
0806 20:29:59.156268 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":526,"threads_started":1}
I20260806 20:29:59.161015 4833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35f35cc07a264e57ab6b38799faefde1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40013 } }
I20260806 20:29:59.161550 4833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:29:59.161669 4833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35f35cc07a264e57ab6b38799faefde1, State: Initialized, Role: FOLLOWER
I20260806 20:29:59.162266 4833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35f35cc07a264e57ab6b38799faefde1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40013 } }
I20260806 20:29:59.162545 4833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:29:59.162700 4833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:29:59.162947 4833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:29:59.165710 4833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35f35cc07a264e57ab6b38799faefde1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40013 } }
I20260806 20:29:59.166327 4833 leader_election.cc:302] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35f35cc07a264e57ab6b38799faefde1; no voters:
I20260806 20:29:59.167019 4833 leader_election.cc:288] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:29:59.167140 4838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:29:59.167939 4838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [term 1 LEADER]: Becoming Leader. State: Replica: 35f35cc07a264e57ab6b38799faefde1, State: Running, Role: LEADER
I20260806 20:29:59.168558 4838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35f35cc07a264e57ab6b38799faefde1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40013 } }
I20260806 20:29:59.169370 4833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:29:59.172042 4839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35f35cc07a264e57ab6b38799faefde1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35f35cc07a264e57ab6b38799faefde1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40013 } } }
I20260806 20:29:59.172646 4839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:59.172179 4840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35f35cc07a264e57ab6b38799faefde1. Latest consensus state: current_term: 1 leader_uuid: "35f35cc07a264e57ab6b38799faefde1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35f35cc07a264e57ab6b38799faefde1" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 40013 } } }
I20260806 20:29:59.172784 4840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1 [sys.catalog]: This master's current role is: LEADER
I20260806 20:29:59.175196 4845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:29:59.181320 4845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:29:59.192878 4845 catalog_manager.cc:1347] Generated new cluster ID: 88ac9d6d19d94b9191e4cea3aa359b34
I20260806 20:29:59.193015 4845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:29:59.210942 4845 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:29:59.212908 4845 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:29:59.227116 4845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1: Generated new TSK 0
I20260806 20:29:59.228236 4845 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:29:59.301282 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40013
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:29:59.651227 4863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:29:59.651603 4863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:29:59.651785 4863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:29:59.659614 4863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:29:59.659798 4863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:29:59.659976 4863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:29:59.670625 4863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:29:59.672156 4863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:29:59.673620 4863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:29:59.687350 4869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:59.687386 4868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:29:59.687641 4871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:29:59.692250 4863 server_base.cc:1034] running on GCE node
I20260806 20:29:59.693046 4863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:29:59.694319 4863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:29:59.695600 4863 hybrid_clock.cc:648] HybridClock initialized: now 1786048199695499 us; error 63 us; skew 500 ppm
I20260806 20:29:59.695958 4863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:29:59 dist-test-slave-6k22 krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048199, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:29:59.699697 4863 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:29:59.701232 4863 webserver.cc:460] Webserver started at http://127.17.109.193:40185/ using document root <none> and password file <none>
I20260806 20:29:59.701951 4863 fs_manager.cc:359] Metadata directory not provided
I20260806 20:29:59.702101 4863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:29:59.702427 4863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:29:59.705479 4863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "617ef9d1ee454394a8594cf85ea371e3"
format_stamp: "Formatted at 2026-08-06 20:29:59 on dist-test-slave-6k22"
server_key: "6d59b90d65cc61320a2548c3d44accb2"
server_key_iv: "fd43e521dad6b014be38c6e54515b4dc"
server_key_version: "encryptionkey@0"
I20260806 20:29:59.706271 4863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "617ef9d1ee454394a8594cf85ea371e3"
format_stamp: "Formatted at 2026-08-06 20:29:59 on dist-test-slave-6k22"
server_key: "6d59b90d65cc61320a2548c3d44accb2"
server_key_iv: "fd43e521dad6b014be38c6e54515b4dc"
server_key_version: "encryptionkey@0"
I20260806 20:29:59.711808 4863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 20:29:59.715299 4879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:59.716638 4863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:29:59.716815 4863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "617ef9d1ee454394a8594cf85ea371e3"
format_stamp: "Formatted at 2026-08-06 20:29:59 on dist-test-slave-6k22"
server_key: "6d59b90d65cc61320a2548c3d44accb2"
server_key_iv: "fd43e521dad6b014be38c6e54515b4dc"
server_key_version: "encryptionkey@0"
I20260806 20:29:59.717015 4863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:29:59.733603 4863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:29:59.737896 4863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:29:59.738202 4863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:29:59.739210 4863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:29:59.740633 4863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:29:59.740749 4863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:59.740902 4863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:29:59.740975 4863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:29:59.776778 4863 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:39229
I20260806 20:29:59.776821 5004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:39229 every 8 connection(s)
I20260806 20:29:59.778241 4863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:29:59.785590 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 4863
I20260806 20:29:59.785794 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:29:59.786355 17847 external_mini_cluster.cc:1428] Setting key 477393274fe64b18200f62e9fe60e698
Aug 06 20:29:59 dist-test-slave-6k22 krb5kdc[4744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048199, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:29:59.799944 5007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:59.781037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:55095 (local address 127.17.109.254:40013)
0806 20:29:59.781515 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0806 20:29:59.781521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:29:59.782388 (+ 867us) server_negotiation.cc:407] Connection header received
0806 20:29:59.784337 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:29:59.784343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:29:59.784443 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:29:59.784567 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:29:59.786228 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:59.786983 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:59.788656 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:29:59.789063 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:29:59.789215 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:59.793189 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:29:59.793194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:29:59.793197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:29:59.793230 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:29:59.796080 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:59.797219 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:59.797223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:59.797226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:59.797347 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:29:59.797888 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:29:59.797892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:29:59.797894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:29:59.798147 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:29:59.798282 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 20:29:59.799417 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0806 20:29:59.799670 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":253,"threads_started":1}
I20260806 20:29:59.801522 5008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:29:59.781392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40013 (local address 127.17.109.193:55095)
0806 20:29:59.782214 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0806 20:29:59.782253 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 20:29:59.784067 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:29:59.784818 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:29:59.784826 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:29:59.785371 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:29:59.786025 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:59.786044 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:59.787157 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:59.787162 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:59.788499 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:29:59.788511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:29:59.789301 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:29:59.789310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:29:59.789546 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:29:59.790234 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:29:59.790262 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:29:59.792927 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:29:59.796231 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:59.796242 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:59.796260 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:59.797032 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:59.797463 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:29:59.797471 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:29:59.797477 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:29:59.797759 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:29:59.798301 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:29:59.798311 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:29:59.798936 (+ 625us) client_negotiation.cc:769] Sending connection context
0806 20:29:59.799294 (+ 358us) client_negotiation.cc:240] Negotiation successful
0806 20:29:59.799947 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":378,"threads_started":1}
I20260806 20:29:59.803213 5005 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40013
I20260806 20:29:59.803535 5005 heartbeater.cc:464] Registering TS with master...
I20260806 20:29:59.804438 5005 heartbeater.cc:511] Master 127.17.109.254:40013 requested a full tablet report, sending...
I20260806 20:29:59.806656 4793 ts_manager.cc:194] Registered new tserver with Master: 617ef9d1ee454394a8594cf85ea371e3 (127.17.109.193:39229)
I20260806 20:29:59.808274 4793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:55095
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:29:59.859035 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40013
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:00.201900 5011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:00.202262 5011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:00.202443 5011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:00.209795 5011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:00.209972 5011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:00.210175 5011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:30:00.220351 5011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:00.221813 5011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:00.223184 5011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:00.237210 5019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:00.237510 5016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:00.237249 5017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:00.238451 5011 server_base.cc:1034] running on GCE node
I20260806 20:30:00.239099 5011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:00.240283 5011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:00.241518 5011 hybrid_clock.cc:648] HybridClock initialized: now 1786048200241467 us; error 41 us; skew 500 ppm
I20260806 20:30:00.241927 5011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:00 dist-test-slave-6k22 krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048200, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:00.245523 5011 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:30:00.247234 5011 webserver.cc:460] Webserver started at http://127.17.109.194:43541/ using document root <none> and password file <none>
I20260806 20:30:00.248106 5011 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:00.248281 5011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:00.248647 5011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:00.252214 5011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "4589822baf644fe889f2d5a77a49df4d"
format_stamp: "Formatted at 2026-08-06 20:30:00 on dist-test-slave-6k22"
server_key: "2213bca5099ecefcbd840be0a8ede1eb"
server_key_iv: "92870dd2e4e462e9a1d6d7d365977582"
server_key_version: "encryptionkey@0"
I20260806 20:30:00.253105 5011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "4589822baf644fe889f2d5a77a49df4d"
format_stamp: "Formatted at 2026-08-06 20:30:00 on dist-test-slave-6k22"
server_key: "2213bca5099ecefcbd840be0a8ede1eb"
server_key_iv: "92870dd2e4e462e9a1d6d7d365977582"
server_key_version: "encryptionkey@0"
I20260806 20:30:00.259163 5011 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 20:30:00.263168 5027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:00.264698 5011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 20:30:00.264904 5011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "4589822baf644fe889f2d5a77a49df4d"
format_stamp: "Formatted at 2026-08-06 20:30:00 on dist-test-slave-6k22"
server_key: "2213bca5099ecefcbd840be0a8ede1eb"
server_key_iv: "92870dd2e4e462e9a1d6d7d365977582"
server_key_version: "encryptionkey@0"
I20260806 20:30:00.265154 5011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:00.280567 5011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:00.285264 5011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:00.285629 5011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:00.286777 5011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:00.288368 5011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:00.288486 5011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:00.288662 5011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:00.288751 5011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:00.324524 5011 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:45129
I20260806 20:30:00.324616 5157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:45129 every 8 connection(s)
I20260806 20:30:00.325878 5011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:30:00.331990 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 5011
I20260806 20:30:00.332177 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:30:00.332621 17847 external_mini_cluster.cc:1428] Setting key 0839968f23b4e4d697ae21ca82c7cbc1
Aug 06 20:30:00 dist-test-slave-6k22 krb5kdc[4744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048200, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:00.350687 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.328994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:48519 (local address 127.17.109.254:40013)
0806 20:30:00.329535 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:00.329542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:00.330481 (+ 939us) server_negotiation.cc:407] Connection header received
0806 20:30:00.332988 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:00.332994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:00.333100 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:00.333232 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:00.335119 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.336366 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.338140 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.338919 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.339058 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:00.342944 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:00.342953 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:00.342961 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:00.343009 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:00.346876 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:00.347814 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:00.347822 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:00.347829 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:00.347999 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:00.348646 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:00.348649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:00.348651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:00.348933 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:00.349045 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:00.350033 (+ 988us) server_negotiation.cc:299] Negotiation successful
0806 20:30:00.350288 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":268,"threads_started":1}
I20260806 20:30:00.351811 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.329418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40013 (local address 127.17.109.194:48519)
0806 20:30:00.330281 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:00.330336 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:00.332581 (+ 2245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:00.333581 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:00.333591 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:00.334128 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:00.334939 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:00.334960 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.336541 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.336545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:00.337982 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:00.337995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.339143 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.339148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:00.339367 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:00.340218 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:00.340243 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:00.342686 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:00.347051 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:00.347058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:00.347073 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:00.347643 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:00.348187 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:00.348195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:00.348200 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:00.348514 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:00.349072 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:00.349082 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:00.349601 (+ 519us) client_negotiation.cc:769] Sending connection context
0806 20:30:00.349929 (+ 328us) client_negotiation.cc:240] Negotiation successful
0806 20:30:00.350515 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":337,"threads_started":1}
I20260806 20:30:00.353627 5158 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40013
I20260806 20:30:00.354049 5158 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:00.354889 5158 heartbeater.cc:511] Master 127.17.109.254:40013 requested a full tablet report, sending...
I20260806 20:30:00.356691 4793 ts_manager.cc:194] Registered new tserver with Master: 4589822baf644fe889f2d5a77a49df4d (127.17.109.194:45129)
I20260806 20:30:00.358026 4793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:48519
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:30:00.407315 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40013
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:38629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:00.766072 5164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:00.766367 5164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:00.766463 5164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:00.774341 5164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:00.774461 5164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:00.774576 5164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:30:00.784811 5164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:38629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:40013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:00.786015 5164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:00.787326 5164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:00.800580 5172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:00.800570 5170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:00.800968 5169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:00.801549 5164 server_base.cc:1034] running on GCE node
I20260806 20:30:00.802119 5164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:00.803066 5164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:00.804294 5164 hybrid_clock.cc:648] HybridClock initialized: now 1786048200804245 us; error 46 us; skew 500 ppm
I20260806 20:30:00.804600 5164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:00 dist-test-slave-6k22 krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048200, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:00.807811 5164 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:30:00.809310 5164 webserver.cc:460] Webserver started at http://127.17.109.195:36067/ using document root <none> and password file <none>
I20260806 20:30:00.810012 5164 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:00.810169 5164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:00.810552 5164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:00.812050 5005 heartbeater.cc:503] Master 127.17.109.254:40013 was elected leader, sending a full tablet report...
I20260806 20:30:00.814134 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca"
format_stamp: "Formatted at 2026-08-06 20:30:00 on dist-test-slave-6k22"
server_key: "ea4c3298d40f08b46cd5a9d49ad5272e"
server_key_iv: "3e9e28cc8f0ebee263b6771abe49ee71"
server_key_version: "encryptionkey@0"
I20260806 20:30:00.814942 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca"
format_stamp: "Formatted at 2026-08-06 20:30:00 on dist-test-slave-6k22"
server_key: "ea4c3298d40f08b46cd5a9d49ad5272e"
server_key_iv: "3e9e28cc8f0ebee263b6771abe49ee71"
server_key_version: "encryptionkey@0"
I20260806 20:30:00.820477 5164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 20:30:00.824374 5182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:00.826032 5164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 20:30:00.826198 5164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca"
format_stamp: "Formatted at 2026-08-06 20:30:00 on dist-test-slave-6k22"
server_key: "ea4c3298d40f08b46cd5a9d49ad5272e"
server_key_iv: "3e9e28cc8f0ebee263b6771abe49ee71"
server_key_version: "encryptionkey@0"
I20260806 20:30:00.826433 5164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:00.867107 5164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:00.872540 5164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:00.872939 5164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:00.873979 5164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:00.875367 5164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:00.875471 5164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:00.875633 5164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:00.875707 5164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:00.910529 5164 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:36791
I20260806 20:30:00.910634 5307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:36791 every 8 connection(s)
I20260806 20:30:00.911904 5164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:30:00.912529 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 5164
I20260806 20:30:00.912737 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:30:00.913321 17847 external_mini_cluster.cc:1428] Setting key c06618b2fe25229e46ff83feb0ff0d04
Aug 06 20:30:00 dist-test-slave-6k22 krb5kdc[4744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048200, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:00.934885 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.914725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:60353 (local address 127.17.109.254:40013)
0806 20:30:00.915190 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:00.915194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:00.916304 (+ 1110us) server_negotiation.cc:407] Connection header received
0806 20:30:00.918923 (+ 2619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:00.918929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:00.919040 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:00.919159 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:00.920750 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.921517 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.923066 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.923465 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.923620 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:00.927364 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:00.927369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:00.927373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:00.927409 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:00.931517 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:00.932446 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:00.932450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:00.932452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:00.932587 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:00.933117 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:00.933121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:00.933123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:00.933443 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:00.933580 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:00.934369 (+ 789us) server_negotiation.cc:299] Negotiation successful
0806 20:30:00.934549 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":243,"threads_started":1}
I20260806 20:30:00.936035 5311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.915339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:40013 (local address 127.17.109.195:60353)
0806 20:30:00.916134 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:00.916201 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:00.918671 (+ 2470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:00.919395 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:00.919403 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:00.919964 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:00.920589 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:00.920607 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.921728 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.921732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:00.922920 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:00.922933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.923720 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.923727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:00.923950 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:00.924676 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:00.924741 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:00.927136 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:00.931740 (+ 4604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:00.931749 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:00.931765 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:00.932306 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:00.932752 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:00.932756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:00.932758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:00.932961 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:00.933591 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:00.933597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:00.933986 (+ 389us) client_negotiation.cc:769] Sending connection context
0806 20:30:00.934239 (+ 253us) client_negotiation.cc:240] Negotiation successful
0806 20:30:00.934695 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":340,"threads_started":1}
I20260806 20:30:00.937537 5308 heartbeater.cc:347] Connected to a master server at 127.17.109.254:40013
I20260806 20:30:00.937917 5308 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:00.938740 5308 heartbeater.cc:511] Master 127.17.109.254:40013 requested a full tablet report, sending...
I20260806 20:30:00.940378 4793 ts_manager.cc:194] Registered new tserver with Master: acd7d33ca24c4337b1c2a8ff42e0e5ca (127.17.109.195:36791)
I20260806 20:30:00.941509 4793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:60353
I20260806 20:30:00.943277 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:30:00.962390 5310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.947432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51908 (local address 127.17.109.254:40013)
0806 20:30:00.947693 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:00.947702 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:00.948073 (+ 371us) server_negotiation.cc:407] Connection header received
0806 20:30:00.948215 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:00.948218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:00.948303 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:00.948386 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:00.950040 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.950755 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.952373 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.952763 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.952880 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:00.955983 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:00.955988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:00.955991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:00.956026 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:00.959059 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:00.960011 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:00.960017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:00.960021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:00.960167 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:00.960721 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:00.960725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:00.960728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:00.961054 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:00.961350 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:00.961825 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 20:30:00.962061 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 20:30:00.966634 4793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:30:00.974064 4793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:30:01.005525 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.987258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:39229 (local address 127.0.0.1:32972)
0806 20:30:00.992587 (+ 5329us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:00.992608 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:00.992737 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:00.994072 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:00.994084 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:00.994140 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:00.995906 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:00.995926 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.998670 (+ 2744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:00.998680 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.001962 (+ 3282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.001980 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.004199 (+ 2219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.004205 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.004504 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.004588 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 20:30:01.004745 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 20:30:01.004977 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5159,"mutex_wait_us":4,"thread_start_us":465,"threads_started":1}
I20260806 20:30:01.006583 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.987811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32972 (local address 127.17.109.193:39229)
0806 20:30:00.992270 (+ 4459us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:00.992296 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:00.992891 (+ 595us) server_negotiation.cc:407] Connection header received
0806 20:30:00.993035 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:00.993047 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:00.993413 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:00.993661 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:00.996176 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:00.998431 (+ 2255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.002176 (+ 3745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.003952 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.004190 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.004401 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.004863 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.006236 (+ 1373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4217,"thread_start_us":4006,"threads_started":1}
I20260806 20:30:01.009470 5322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.987227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45129 (local address 127.0.0.1:49642)
0806 20:30:00.988212 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:00.988268 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:00.988515 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:01.001055 (+ 12540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:01.001060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:01.001103 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:01.001649 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.001669 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.003845 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.003852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.006512 (+ 2660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.006529 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.008524 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.008531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.008803 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.008839 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 20:30:01.008941 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 20:30:01.009086 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":87,"thread_start_us":5047,"threads_started":1}
I20260806 20:30:01.010772 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.987816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49642 (local address 127.17.109.194:45129)
0806 20:30:01.000235 (+ 12419us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:01.000248 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:01.000290 (+ 42us) server_negotiation.cc:407] Connection header received
0806 20:30:01.000433 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:01.000445 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:01.000804 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:01.001077 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:01.001928 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.003695 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.006683 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.008256 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.008445 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.008617 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.010125 (+ 1508us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.010332 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12144,"thread_start_us":13454,"threads_started":1}
I20260806 20:30:01.010625 4930 tablet_service.cc:1467] Processing CreateTablet for tablet 24f8ef180be8427c80a25a0a65f8e434 (DEFAULT_TABLE table=test-table [id=5930451407754a09a2a4418dda8c8af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:01.012260 4930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24f8ef180be8427c80a25a0a65f8e434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:01.012799 5326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.992722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:36791 (local address 127.0.0.1:53388)
0806 20:30:01.002001 (+ 9279us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:01.002026 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:01.002140 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:01.004491 (+ 2351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:01.004494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:01.004517 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:01.005096 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.005111 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.007081 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.007086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.009067 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.009081 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.011917 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.011923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.012090 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.012123 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 20:30:01.012227 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 20:30:01.012424 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9126,"mutex_wait_us":3,"thread_start_us":8924,"threads_started":1}
I20260806 20:30:01.013139 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:00.993136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.17.109.195:36791)
0806 20:30:01.003511 (+ 10375us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:01.003520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:01.003579 (+ 59us) server_negotiation.cc:407] Connection header received
0806 20:30:01.003689 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:01.003697 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:01.003975 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:01.004185 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:01.005348 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.006930 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.009249 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.011683 (+ 2434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.011947 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.012145 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.012325 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.012573 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10179,"thread_start_us":9987,"threads_started":1}
I20260806 20:30:01.014999 5082 tablet_service.cc:1467] Processing CreateTablet for tablet 24f8ef180be8427c80a25a0a65f8e434 (DEFAULT_TABLE table=test-table [id=5930451407754a09a2a4418dda8c8af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:01.016593 5082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24f8ef180be8427c80a25a0a65f8e434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:01.017033 5233 tablet_service.cc:1467] Processing CreateTablet for tablet 24f8ef180be8427c80a25a0a65f8e434 (DEFAULT_TABLE table=test-table [id=5930451407754a09a2a4418dda8c8af3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:01.018606 5233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24f8ef180be8427c80a25a0a65f8e434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:01.024562 5328 tablet_bootstrap.cc:492] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: Bootstrap starting.
I20260806 20:30:01.027930 5328 tablet_bootstrap.cc:654] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:01.029127 5328 log.cc:824] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:01.032940 5328 tablet_bootstrap.cc:492] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: No bootstrap required, opened a new log
I20260806 20:30:01.033229 5328 ts_tablet_manager.cc:1397] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 20:30:01.033677 5331 tablet_bootstrap.cc:492] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Bootstrap starting.
I20260806 20:30:01.035200 5332 tablet_bootstrap.cc:492] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Bootstrap starting.
I20260806 20:30:01.037488 5331 tablet_bootstrap.cc:654] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:01.038836 5331 log.cc:824] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:01.039064 5332 tablet_bootstrap.cc:654] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:01.040503 5332 log.cc:824] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:01.042327 5331 tablet_bootstrap.cc:492] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: No bootstrap required, opened a new log
I20260806 20:30:01.042601 5331 ts_tablet_manager.cc:1397] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260806 20:30:01.043439 5332 tablet_bootstrap.cc:492] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: No bootstrap required, opened a new log
I20260806 20:30:01.043768 5332 ts_tablet_manager.cc:1397] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260806 20:30:01.043642 5328 raft_consensus.cc:348] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.044117 5328 raft_consensus.cc:374] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:01.044224 5328 raft_consensus.cc:729] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617ef9d1ee454394a8594cf85ea371e3, State: Initialized, Role: FOLLOWER
I20260806 20:30:01.044935 5328 consensus_queue.cc:260] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.049059 5328 ts_tablet_manager.cc:1428] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260806 20:30:01.052433 5331 raft_consensus.cc:348] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.052933 5331 raft_consensus.cc:374] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:01.053042 5331 raft_consensus.cc:729] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4589822baf644fe889f2d5a77a49df4d, State: Initialized, Role: FOLLOWER
I20260806 20:30:01.053682 5331 consensus_queue.cc:260] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.054913 5158 heartbeater.cc:503] Master 127.17.109.254:40013 was elected leader, sending a full tablet report...
I20260806 20:30:01.055696 5331 ts_tablet_manager.cc:1428] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Time spent starting tablet: real 0.013s user 0.012s sys 0.003s
I20260806 20:30:01.056789 5332 raft_consensus.cc:348] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.057294 5332 raft_consensus.cc:374] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:01.057422 5332 raft_consensus.cc:729] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acd7d33ca24c4337b1c2a8ff42e0e5ca, State: Initialized, Role: FOLLOWER
I20260806 20:30:01.058157 5332 consensus_queue.cc:260] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.059394 5308 heartbeater.cc:503] Master 127.17.109.254:40013 was elected leader, sending a full tablet report...
I20260806 20:30:01.060091 5332 ts_tablet_manager.cc:1428] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Time spent starting tablet: real 0.016s user 0.006s sys 0.009s
W20260806 20:30:01.078110 5159 tablet.cc:2369] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:01.160399 5337 raft_consensus.cc:482] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:30:01.160795 5337 raft_consensus.cc:504] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.163914 5337 leader_election.cc:288] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 617ef9d1ee454394a8594cf85ea371e3 (127.17.109.193:39229), acd7d33ca24c4337b1c2a8ff42e0e5ca (127.17.109.195:36791)
W20260806 20:30:01.164062 5309 tablet.cc:2369] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:01.171228 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:01.163668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:48253 (local address 127.17.109.193:39229)
0806 20:30:01.163931 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:01.163941 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:01.164026 (+ 85us) server_negotiation.cc:407] Connection header received
0806 20:30:01.164357 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:01.164368 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:01.164477 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:01.164602 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:01.165480 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.166449 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.169013 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.170013 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.170199 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.170307 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.170845 (+ 538us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.170980 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 20:30:01.171435 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:01.163359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:40429 (local address 127.17.109.195:36791)
0806 20:30:01.163765 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:01.163771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:01.164796 (+ 1025us) server_negotiation.cc:407] Connection header received
0806 20:30:01.165106 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:01.165126 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:01.165228 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:01.165352 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:01.165961 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.166950 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.168914 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.169968 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.170258 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.170368 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.170795 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.170953 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260806 20:30:01.171499 5340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:01.163300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:36791 (local address 127.17.109.194:40429)
0806 20:30:01.164703 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:01.164721 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:01.164915 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:01.165370 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:01.165373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:01.165425 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:01.165795 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.165808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.167114 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.167118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.168706 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.168718 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.170300 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.170308 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.170619 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.170646 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 20:30:01.170723 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 20:30:01.170867 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1270,"mutex_wait_us":91,"thread_start_us":1073,"threads_started":1}
I20260806 20:30:01.171808 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:01.163162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:39229 (local address 127.17.109.194:48253)
0806 20:30:01.163928 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:01.163948 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:01.164110 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:01.164816 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:01.164820 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:01.164848 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:01.165294 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.165313 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.166603 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.166610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.168702 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:01.168716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.170322 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.170331 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:01.170633 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.170665 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 20:30:01.170806 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 20:30:01.170952 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":9,"thread_start_us":425,"threads_started":1}
I20260806 20:30:01.172554 5256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f8ef180be8427c80a25a0a65f8e434" candidate_uuid: "4589822baf644fe889f2d5a77a49df4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" is_pre_election: true
I20260806 20:30:01.172982 4952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f8ef180be8427c80a25a0a65f8e434" candidate_uuid: "4589822baf644fe889f2d5a77a49df4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617ef9d1ee454394a8594cf85ea371e3" is_pre_election: true
I20260806 20:30:01.173192 5256 raft_consensus.cc:2454] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4589822baf644fe889f2d5a77a49df4d in term 0.
I20260806 20:30:01.173592 4952 raft_consensus.cc:2454] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4589822baf644fe889f2d5a77a49df4d in term 0.
I20260806 20:30:01.173923 5031 leader_election.cc:302] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4589822baf644fe889f2d5a77a49df4d, acd7d33ca24c4337b1c2a8ff42e0e5ca; no voters:
I20260806 20:30:01.174360 5337 raft_consensus.cc:2781] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:30:01.174512 5337 raft_consensus.cc:482] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:30:01.174638 5337 raft_consensus.cc:3037] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:01.176955 5337 raft_consensus.cc:504] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.177627 5337 leader_election.cc:288] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [CANDIDATE]: Term 1 election: Requested vote from peers 617ef9d1ee454394a8594cf85ea371e3 (127.17.109.193:39229), acd7d33ca24c4337b1c2a8ff42e0e5ca (127.17.109.195:36791)
I20260806 20:30:01.178094 4952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f8ef180be8427c80a25a0a65f8e434" candidate_uuid: "4589822baf644fe889f2d5a77a49df4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "617ef9d1ee454394a8594cf85ea371e3"
I20260806 20:30:01.178193 5256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24f8ef180be8427c80a25a0a65f8e434" candidate_uuid: "4589822baf644fe889f2d5a77a49df4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca"
I20260806 20:30:01.178326 4952 raft_consensus.cc:3037] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:01.178398 5256 raft_consensus.cc:3037] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:01.181717 4952 raft_consensus.cc:2454] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4589822baf644fe889f2d5a77a49df4d in term 1.
I20260806 20:30:01.181705 5256 raft_consensus.cc:2454] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4589822baf644fe889f2d5a77a49df4d in term 1.
I20260806 20:30:01.182154 5030 leader_election.cc:302] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4589822baf644fe889f2d5a77a49df4d, 617ef9d1ee454394a8594cf85ea371e3; no voters:
I20260806 20:30:01.182482 5337 raft_consensus.cc:2781] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:01.182945 5337 raft_consensus.cc:686] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 1 LEADER]: Becoming Leader. State: Replica: 4589822baf644fe889f2d5a77a49df4d, State: Running, Role: LEADER
I20260806 20:30:01.183697 5337 consensus_queue.cc:237] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } }
I20260806 20:30:01.188424 4792 catalog_manager.cc:5495] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4589822baf644fe889f2d5a77a49df4d (127.17.109.194). New cstate: current_term: 1 leader_uuid: "4589822baf644fe889f2d5a77a49df4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4589822baf644fe889f2d5a77a49df4d" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 } health_report { overall_health: UNKNOWN } } }
I20260806 20:30:01.212380 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:01.206931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49646 (local address 127.17.109.194:45129)
0806 20:30:01.207110 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:01.207114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:01.207275 (+ 161us) server_negotiation.cc:407] Connection header received
0806 20:30:01.207438 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:01.207441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:01.207513 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:01.207726 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:01.208166 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.208932 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.210237 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.210633 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.210787 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.211242 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:01.211851 (+ 609us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:01.211993 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.212126 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.212166 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 20:30:01.221069 4952 raft_consensus.cc:1261] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 1 FOLLOWER]: Refusing update from remote peer 4589822baf644fe889f2d5a77a49df4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:01.221069 5256 raft_consensus.cc:1261] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 1 FOLLOWER]: Refusing update from remote peer 4589822baf644fe889f2d5a77a49df4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:01.222198 5341 consensus_queue.cc:1035] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "617ef9d1ee454394a8594cf85ea371e3" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 39229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:01.222561 5337 consensus_queue.cc:1035] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "acd7d33ca24c4337b1c2a8ff42e0e5ca" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 36791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:01.235399 5345 mvcc.cc:205] Tried to move back new op lower bound from 7315653432189644800 to 7315653432052285440. Current Snapshot: MvccSnapshot[applied={T|T < 7315653432189644800}]
I20260806 20:30:01.237919 5346 mvcc.cc:205] Tried to move back new op lower bound from 7315653432189644800 to 7315653432052285440. Current Snapshot: MvccSnapshot[applied={T|T < 7315653432189644800}]
I20260806 20:30:01.255229 4793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:30:01.255643 4793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:30:01.260161 4793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35f35cc07a264e57ab6b38799faefde1: Sending DeleteTablet for 3 replicas of tablet 24f8ef180be8427c80a25a0a65f8e434
I20260806 20:30:01.261620 5233 tablet_service.cc:1514] Processing DeleteTablet for tablet 24f8ef180be8427c80a25a0a65f8e434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:01 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:53388
I20260806 20:30:01.261679 4930 tablet_service.cc:1514] Processing DeleteTablet for tablet 24f8ef180be8427c80a25a0a65f8e434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:01 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:32972
I20260806 20:30:01.261826 5082 tablet_service.cc:1514] Processing DeleteTablet for tablet 24f8ef180be8427c80a25a0a65f8e434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:01 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:49642
I20260806 20:30:01.264523 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 4863
I20260806 20:30:01.283528 5356 tablet_replica.cc:330] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: stopping tablet replica
I20260806 20:30:01.284919 5357 tablet_replica.cc:330] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: stopping tablet replica
I20260806 20:30:01.284320 5358 tablet_replica.cc:330] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: stopping tablet replica
I20260806 20:30:01.285446 5357 raft_consensus.cc:2229] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:30:01.286087 5357 raft_consensus.cc:2258] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:01.286399 5356 raft_consensus.cc:2229] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:30:01.286968 5356 raft_consensus.cc:2258] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:01.287165 5358 raft_consensus.cc:2229] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:30:01.288424 5357 ts_tablet_manager.cc:1905] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:01.288678 5358 raft_consensus.cc:2258] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:01.290230 5356 ts_tablet_manager.cc:1905] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:01.295580 5357 ts_tablet_manager.cc:1918] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:01.295753 5357 log.cc:1196] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/24f8ef180be8427c80a25a0a65f8e434
I20260806 20:30:01.296324 5357 ts_tablet_manager.cc:1939] T 24f8ef180be8427c80a25a0a65f8e434 P 617ef9d1ee454394a8594cf85ea371e3: Deleting consensus metadata
I20260806 20:30:01.296375 5358 ts_tablet_manager.cc:1905] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 06 20:30:01 dist-test-slave-6k22 krb5kdc[4744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:30:01.298256 4776 catalog_manager.cc:4841] TS 617ef9d1ee454394a8594cf85ea371e3 (127.17.109.193:39229): tablet 24f8ef180be8427c80a25a0a65f8e434 (table test-table [id=5930451407754a09a2a4418dda8c8af3]) successfully deleted
I20260806 20:30:01.303320 5356 ts_tablet_manager.cc:1918] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:01.303514 5356 log.cc:1196] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/24f8ef180be8427c80a25a0a65f8e434
I20260806 20:30:01.304215 5356 ts_tablet_manager.cc:1939] T 24f8ef180be8427c80a25a0a65f8e434 P acd7d33ca24c4337b1c2a8ff42e0e5ca: Deleting consensus metadata
I20260806 20:30:01.305225 5358 ts_tablet_manager.cc:1918] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:01.305411 5358 log.cc:1196] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/24f8ef180be8427c80a25a0a65f8e434
I20260806 20:30:01.306020 5358 ts_tablet_manager.cc:1939] T 24f8ef180be8427c80a25a0a65f8e434 P 4589822baf644fe889f2d5a77a49df4d: Deleting consensus metadata
I20260806 20:30:01.306365 4777 catalog_manager.cc:4841] TS acd7d33ca24c4337b1c2a8ff42e0e5ca (127.17.109.195:36791): tablet 24f8ef180be8427c80a25a0a65f8e434 (table test-table [id=5930451407754a09a2a4418dda8c8af3]) successfully deleted
I20260806 20:30:01.307729 4777 catalog_manager.cc:4841] TS 4589822baf644fe889f2d5a77a49df4d (127.17.109.194:45129): tablet 24f8ef180be8427c80a25a0a65f8e434 (table test-table [id=5930451407754a09a2a4418dda8c8af3]) successfully deleted
I20260806 20:30:01.308300 5325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:01.265343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.17.109.193:39229)
0806 20:30:01.265549 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:01.265558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:01.270892 (+ 5334us) server_negotiation.cc:407] Connection header received
0806 20:30:01.271812 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:01.271822 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:01.271929 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:01.272027 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:01.276548 (+ 4521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.278197 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.284448 (+ 6251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.285222 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.285358 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.299384 (+ 14026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:01.299398 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:01.299417 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:01.299461 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:01.303522 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:01.304698 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:01.304708 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:01.304712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:01.304897 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:01.305420 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:01.305425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:01.305428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:01.305950 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:01.306312 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.307642 (+ 1330us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.307878 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 20:30:01.309130 4997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:01.424818 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 4863
I20260806 20:30:01.445539 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 5011
Aug 06 20:30:01 dist-test-slave-6k22 krb5kdc[4744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:30:01.464327 5324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:01.446570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49650 (local address 127.17.109.194:45129)
0806 20:30:01.446786 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:01.446792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:01.446813 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:30:01.446881 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:01.446886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:01.446981 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:01.447095 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:01.448523 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.449774 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.451092 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.451964 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.452100 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.457025 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:01.457044 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:01.457072 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:01.457122 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:01.461214 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:01.462030 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:01.462039 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:01.462043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:01.462225 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:01.462661 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:01.462665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:01.462668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:01.463135 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:01.463492 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.463687 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.463919 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:30:01.465204 5152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:01.581969 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 5011
I20260806 20:30:01.603533 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 5164
Aug 06 20:30:01 dist-test-slave-6k22 krb5kdc[4744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:30:01.622182 5327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:01.604548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53394 (local address 127.17.109.195:36791)
0806 20:30:01.604804 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:01.604809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:01.604833 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:30:01.604905 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:01.604910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:01.604983 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:01.605150 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:01.606533 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.607355 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.609032 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:01.609487 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:01.609604 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:01.614668 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:01.614688 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:01.614706 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:01.614756 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:01.618840 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:01.619817 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:01.619825 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:01.619828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:01.619995 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:01.620432 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:01.620436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:01.620439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:01.620877 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:01.621144 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:01.621489 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 20:30:01.621699 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 20:30:01.623036 5303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:01.714855 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 5164
I20260806 20:30:01.734352 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 4757
I20260806 20:30:01.735528 4828 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:01.847847 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 4757
2026-08-06T20:30:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4766 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T20:30:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:01Z Disabled control of system clock
I20260806 20:30:01.939810 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:42355
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:42767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:42355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 20:30:02.284505 5368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:02.284811 5368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:02.284893 5368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:02.292011 5368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:02.292122 5368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:02.292166 5368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:02.292225 5368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:02.292276 5368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:02.301337 5368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:42355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:42355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:02.302595 5368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:02.304355 5368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:02.317313 5373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:02.317360 5374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:02.317265 5376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:02.318797 5368 server_base.cc:1034] running on GCE node
I20260806 20:30:02.319707 5368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:02.321236 5368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:02.322504 5368 hybrid_clock.cc:648] HybridClock initialized: now 1786048202322452 us; error 49 us; skew 500 ppm
I20260806 20:30:02.322930 5368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:30:02.325533 5368 webserver.cc:460] Webserver started at http://127.17.109.254:41057/ using document root <none> and password file <none>
I20260806 20:30:02.326252 5368 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:02.326391 5368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:02.326721 5368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:02.329805 5368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "8f9396649bc24ffea5556621be0eb258"
format_stamp: "Formatted at 2026-08-06 20:30:02 on dist-test-slave-6k22"
server_key: "111cf759a16f8f4a69e35c27cbb58ffb"
server_key_iv: "e6cdcfc903662b33170d5dab6dbf6608"
server_key_version: "encryptionkey@0"
I20260806 20:30:02.330627 5368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "8f9396649bc24ffea5556621be0eb258"
format_stamp: "Formatted at 2026-08-06 20:30:02 on dist-test-slave-6k22"
server_key: "111cf759a16f8f4a69e35c27cbb58ffb"
server_key_iv: "e6cdcfc903662b33170d5dab6dbf6608"
server_key_version: "encryptionkey@0"
I20260806 20:30:02.335927 5368 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 20:30:02.339749 5383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:02.341252 5368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:02.341488 5368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "8f9396649bc24ffea5556621be0eb258"
format_stamp: "Formatted at 2026-08-06 20:30:02 on dist-test-slave-6k22"
server_key: "111cf759a16f8f4a69e35c27cbb58ffb"
server_key_iv: "e6cdcfc903662b33170d5dab6dbf6608"
server_key_version: "encryptionkey@0"
I20260806 20:30:02.341770 5368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:02.369866 5368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:02.370729 5368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:02.371004 5368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:02.391122 5368 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:42355
I20260806 20:30:02.391131 5441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:42355 every 8 connection(s)
I20260806 20:30:02.392505 5368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:02.396394 5442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:02.398794 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 5368
I20260806 20:30:02.398978 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:02.399468 17847 external_mini_cluster.cc:1428] Setting key 3b36dd738b45a56043c9760de19fa5d1
I20260806 20:30:02.410210 5442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258: Bootstrap starting.
I20260806 20:30:02.414475 5442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:02.415956 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:02.401652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.17.109.254:42355)
0806 20:30:02.402878 (+ 1226us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:02.402893 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:02.402945 (+ 52us) server_negotiation.cc:407] Connection header received
0806 20:30:02.405340 (+ 2395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:02.405372 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:02.405999 (+ 627us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:02.406468 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:02.407048 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:02.409137 (+ 2089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:02.410484 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:02.411513 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:02.411725 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:02.412897 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:02.412908 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:02.412925 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:02.413645 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:02.413774 (+ 129us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:02.413801 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:02.413914 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:02.414176 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 20:30:02.414520 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":440,"threads_started":1}
I20260806 20:30:02.415933 5442 log.cc:824] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:02.419487 5442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258: No bootstrap required, opened a new log
I20260806 20:30:02.427093 5442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9396649bc24ffea5556621be0eb258" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42355 } }
I20260806 20:30:02.427454 5442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:02.427531 5442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9396649bc24ffea5556621be0eb258, State: Initialized, Role: FOLLOWER
I20260806 20:30:02.428105 5442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9396649bc24ffea5556621be0eb258" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42355 } }
I20260806 20:30:02.428315 5442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:02.428423 5442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:02.428548 5442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:02.430830 5442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9396649bc24ffea5556621be0eb258" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42355 } }
I20260806 20:30:02.431360 5442 leader_election.cc:302] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f9396649bc24ffea5556621be0eb258; no voters:
I20260806 20:30:02.432003 5442 leader_election.cc:288] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:02.432176 5447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:02.433000 5447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [term 1 LEADER]: Becoming Leader. State: Replica: 8f9396649bc24ffea5556621be0eb258, State: Running, Role: LEADER
I20260806 20:30:02.433585 5447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9396649bc24ffea5556621be0eb258" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42355 } }
I20260806 20:30:02.434170 5442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:02.436496 5448 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f9396649bc24ffea5556621be0eb258" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9396649bc24ffea5556621be0eb258" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42355 } } }
I20260806 20:30:02.436954 5448 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:02.438328 5449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f9396649bc24ffea5556621be0eb258. Latest consensus state: current_term: 1 leader_uuid: "8f9396649bc24ffea5556621be0eb258" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9396649bc24ffea5556621be0eb258" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42355 } } }
I20260806 20:30:02.438715 5449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:02.439162 5457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:02.443674 5457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:02.452888 5457 catalog_manager.cc:1347] Generated new cluster ID: a149327e4a6b4721942f08ae5e7c8951
I20260806 20:30:02.453014 5457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:02.464046 5457 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:02.465903 5457 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:02.476397 5457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258: Generated new TSK 0
I20260806 20:30:02.477460 5457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:02.483192 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42355
--builtin_ntp_servers=127.17.109.212:42767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 20:30:02.830209 5468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:02.830519 5468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:02.830610 5468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:02.838483 5468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:02.838594 5468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:02.838718 5468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:30:02.848533 5468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:02.849814 5468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:02.851616 5468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:02.865223 5473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:02.865280 5474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:02.865309 5476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:02.866593 5468 server_base.cc:1034] running on GCE node
I20260806 20:30:02.867239 5468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:02.868268 5468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:02.869480 5468 hybrid_clock.cc:648] HybridClock initialized: now 1786048202869415 us; error 54 us; skew 500 ppm
I20260806 20:30:02.869799 5468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:30:02.872061 5468 webserver.cc:460] Webserver started at http://127.17.109.193:45239/ using document root <none> and password file <none>
I20260806 20:30:02.872745 5468 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:02.872859 5468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:02.873184 5468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:02.876235 5468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "cae884863896410ab3be9ff9c925f60b"
format_stamp: "Formatted at 2026-08-06 20:30:02 on dist-test-slave-6k22"
server_key: "17c8a66541b993630d4b42eaa05484ac"
server_key_iv: "1bc94609976c3853dd8cd376493a05bb"
server_key_version: "encryptionkey@0"
I20260806 20:30:02.877051 5468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "cae884863896410ab3be9ff9c925f60b"
format_stamp: "Formatted at 2026-08-06 20:30:02 on dist-test-slave-6k22"
server_key: "17c8a66541b993630d4b42eaa05484ac"
server_key_iv: "1bc94609976c3853dd8cd376493a05bb"
server_key_version: "encryptionkey@0"
I20260806 20:30:02.882176 5468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:30:02.885910 5483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:02.887275 5468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:30:02.887461 5468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "cae884863896410ab3be9ff9c925f60b"
format_stamp: "Formatted at 2026-08-06 20:30:02 on dist-test-slave-6k22"
server_key: "17c8a66541b993630d4b42eaa05484ac"
server_key_iv: "1bc94609976c3853dd8cd376493a05bb"
server_key_version: "encryptionkey@0"
I20260806 20:30:02.887706 5468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:02.906625 5468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:02.907460 5468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:02.907822 5468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:02.908818 5468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:02.910292 5468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:02.910413 5468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:02.910549 5468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:02.910660 5468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:02.946722 5468 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:45167
I20260806 20:30:02.946784 5605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:45167 every 8 connection(s)
I20260806 20:30:02.948319 5468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:30:02.955750 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 5468
I20260806 20:30:02.955966 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:30:02.956533 17847 external_mini_cluster.cc:1428] Setting key 3de28c4f6b93b949276168c08a7eae86
I20260806 20:30:02.959520 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42355
--builtin_ntp_servers=127.17.109.212:42767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 20:30:02.962138 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:02.951142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:34541 (local address 127.17.109.254:42355)
0806 20:30:02.951692 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:02.951699 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:02.952822 (+ 1123us) server_negotiation.cc:407] Connection header received
0806 20:30:02.954695 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:02.954699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:02.954768 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:02.954869 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:02.956031 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:02.957101 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:02.958949 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:02.959650 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:02.959800 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:02.961183 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:02.961191 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:02.961198 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:02.961223 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:02.961280 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:02.961299 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:02.961393 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:02.961607 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 20:30:02.961710 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":228,"threads_started":1}
I20260806 20:30:02.963035 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:02.951742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:42355 (local address 127.17.109.193:34541)
0806 20:30:02.952664 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:02.952701 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:02.954406 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:02.955060 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:02.955067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:02.955155 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:02.955829 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:02.955848 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:02.957279 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:02.957284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:02.958806 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:02.958818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:02.959938 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:02.959944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:02.960138 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:02.960863 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:30:02.960866 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:02.960904 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:30:02.960982 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:02.961408 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:02.961417 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:02.961423 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 20:30:02.961566 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 20:30:02.961894 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":269,"threads_started":1}
I20260806 20:30:02.964694 5606 heartbeater.cc:347] Connected to a master server at 127.17.109.254:42355
I20260806 20:30:02.965011 5606 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:02.965927 5606 heartbeater.cc:511] Master 127.17.109.254:42355 requested a full tablet report, sending...
I20260806 20:30:02.968389 5403 ts_manager.cc:194] Registered new tserver with Master: cae884863896410ab3be9ff9c925f60b (127.17.109.193:45167)
I20260806 20:30:02.970551 5403 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.193:34541
W20260806 20:30:03.301723 5610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:03.302016 5610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:03.302157 5610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:03.309542 5610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:03.309659 5610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:03.309834 5610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:30:03.319095 5610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:03.320497 5610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:03.322294 5610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:03.336053 5620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:03.336025 5617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:03.336005 5618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:03.337054 5610 server_base.cc:1034] running on GCE node
I20260806 20:30:03.337755 5610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:03.338840 5610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:03.340075 5610 hybrid_clock.cc:648] HybridClock initialized: now 1786048203340028 us; error 38 us; skew 500 ppm
I20260806 20:30:03.340427 5610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:30:03.342911 5610 webserver.cc:460] Webserver started at http://127.17.109.194:43541/ using document root <none> and password file <none>
I20260806 20:30:03.343686 5610 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:03.343807 5610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:03.344156 5610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:03.347600 5610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "48d23afea9c64e80bd2f5623ff446a18"
format_stamp: "Formatted at 2026-08-06 20:30:03 on dist-test-slave-6k22"
server_key: "c8f2579e65dd411db40c7702aa374cbb"
server_key_iv: "4229b3277921cb56854117a0645748ca"
server_key_version: "encryptionkey@0"
I20260806 20:30:03.348448 5610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "48d23afea9c64e80bd2f5623ff446a18"
format_stamp: "Formatted at 2026-08-06 20:30:03 on dist-test-slave-6k22"
server_key: "c8f2579e65dd411db40c7702aa374cbb"
server_key_iv: "4229b3277921cb56854117a0645748ca"
server_key_version: "encryptionkey@0"
I20260806 20:30:03.353750 5610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:30:03.357574 5627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:03.359033 5610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:30:03.359215 5610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "48d23afea9c64e80bd2f5623ff446a18"
format_stamp: "Formatted at 2026-08-06 20:30:03 on dist-test-slave-6k22"
server_key: "c8f2579e65dd411db40c7702aa374cbb"
server_key_iv: "4229b3277921cb56854117a0645748ca"
server_key_version: "encryptionkey@0"
I20260806 20:30:03.359449 5610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:03.384374 5610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:03.385505 5610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:03.385838 5610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:03.387029 5610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:03.388701 5610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:03.388844 5610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:03.388955 5610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:03.389051 5610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:03.426370 5610 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:45481
I20260806 20:30:03.426473 5748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:45481 every 8 connection(s)
I20260806 20:30:03.427856 5610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:30:03.433598 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 5610
I20260806 20:30:03.433786 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:30:03.434306 17847 external_mini_cluster.cc:1428] Setting key e2d87db44ff76b379e265d28801d6691
I20260806 20:30:03.437561 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42355
--builtin_ntp_servers=127.17.109.212:42767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 20:30:03.441912 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.430508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:60605 (local address 127.17.109.254:42355)
0806 20:30:03.430714 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:03.430719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:03.432174 (+ 1455us) server_negotiation.cc:407] Connection header received
0806 20:30:03.434188 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:03.434194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:03.434291 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:03.434417 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:03.435854 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.436860 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.438681 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.439445 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.439618 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.440955 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:03.440963 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:03.440970 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:03.441009 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:03.441053 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:03.441071 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:03.441166 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:03.441404 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 20:30:03.441519 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 20:30:03.442775 5751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.431054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:42355 (local address 127.17.109.194:60605)
0806 20:30:03.432011 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:03.432056 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:03.433908 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:03.434692 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:03.434701 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:03.434880 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:03.435686 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.435710 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.437050 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.437054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.438532 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.438545 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.439703 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.439708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.439913 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.440633 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:30:03.440635 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:03.440676 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:30:03.440759 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:03.441173 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:03.441181 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:03.441186 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 20:30:03.441345 (+ 159us) client_negotiation.cc:240] Negotiation successful
0806 20:30:03.441649 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":270,"threads_started":1}
I20260806 20:30:03.444255 5749 heartbeater.cc:347] Connected to a master server at 127.17.109.254:42355
I20260806 20:30:03.444589 5749 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:03.445554 5749 heartbeater.cc:511] Master 127.17.109.254:42355 requested a full tablet report, sending...
I20260806 20:30:03.447135 5403 ts_manager.cc:194] Registered new tserver with Master: 48d23afea9c64e80bd2f5623ff446a18 (127.17.109.194:45481)
I20260806 20:30:03.448041 5403 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.194:60605
W20260806 20:30:03.778234 5752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:03.778609 5752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:03.778774 5752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:03.786345 5752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:03.786480 5752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:03.786657 5752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:30:03.796216 5752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:42355
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:03.797578 5752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:03.799412 5752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:03.812686 5761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:03.812772 5757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:03.812747 5759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:03.813261 5752 server_base.cc:1034] running on GCE node
I20260806 20:30:03.814042 5752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:03.815064 5752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:03.816331 5752 hybrid_clock.cc:648] HybridClock initialized: now 1786048203816274 us; error 49 us; skew 500 ppm
I20260806 20:30:03.816694 5752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:30:03.819005 5752 webserver.cc:460] Webserver started at http://127.17.109.195:40939/ using document root <none> and password file <none>
I20260806 20:30:03.819820 5752 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:03.819957 5752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:03.820277 5752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:03.823136 5752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "c4fd17526c1c4a92a59c6d9bef4dd891"
format_stamp: "Formatted at 2026-08-06 20:30:03 on dist-test-slave-6k22"
server_key: "86471ea84f9f8249b0e1ba0223dbd72f"
server_key_iv: "3ff6580d3d24372745524de01fc3576b"
server_key_version: "encryptionkey@0"
I20260806 20:30:03.823980 5752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "c4fd17526c1c4a92a59c6d9bef4dd891"
format_stamp: "Formatted at 2026-08-06 20:30:03 on dist-test-slave-6k22"
server_key: "86471ea84f9f8249b0e1ba0223dbd72f"
server_key_iv: "3ff6580d3d24372745524de01fc3576b"
server_key_version: "encryptionkey@0"
I20260806 20:30:03.829103 5752 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:30:03.832684 5768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:03.834098 5752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:30:03.834295 5752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "c4fd17526c1c4a92a59c6d9bef4dd891"
format_stamp: "Formatted at 2026-08-06 20:30:03 on dist-test-slave-6k22"
server_key: "86471ea84f9f8249b0e1ba0223dbd72f"
server_key_iv: "3ff6580d3d24372745524de01fc3576b"
server_key_version: "encryptionkey@0"
I20260806 20:30:03.834518 5752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:03.855172 5752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:03.856056 5752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:03.856374 5752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:03.857328 5752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:03.858783 5752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:03.858906 5752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:03.859032 5752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:03.859120 5752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:03.894238 5752 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:33399
I20260806 20:30:03.894326 5897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:33399 every 8 connection(s)
I20260806 20:30:03.895532 5752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:30:03.902232 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 5752
I20260806 20:30:03.902434 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:30:03.903047 17847 external_mini_cluster.cc:1428] Setting key ac6d348265b5a8639acb902809f1fd05
I20260806 20:30:03.909533 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.898306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:50689 (local address 127.17.109.254:42355)
0806 20:30:03.898514 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:03.898519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:03.899930 (+ 1411us) server_negotiation.cc:407] Connection header received
0806 20:30:03.901921 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:03.901926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:03.901996 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:03.902086 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:03.903671 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.904652 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.906794 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.907226 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.907324 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.908835 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:03.908840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:03.908843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:03.908860 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:03.908891 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:03.908902 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:03.908972 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:03.909225 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 20:30:03.909299 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 20:30:03.910532 5900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.898789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:42355 (local address 127.17.109.195:50689)
0806 20:30:03.899766 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:03.899812 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:03.901593 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:03.902384 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:03.902398 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:03.902527 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:03.903389 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.903417 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.904819 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.904826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.906661 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.906673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.907419 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.907424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.907766 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.908495 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:30:03.908498 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:03.908537 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:30:03.908616 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:03.909011 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:03.909020 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:03.909027 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 20:30:03.909188 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 20:30:03.909489 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":293,"threads_started":1}
I20260806 20:30:03.911900 5898 heartbeater.cc:347] Connected to a master server at 127.17.109.254:42355
I20260806 20:30:03.912158 5898 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:03.912843 5898 heartbeater.cc:511] Master 127.17.109.254:42355 requested a full tablet report, sending...
I20260806 20:30:03.914202 5403 ts_manager.cc:194] Registered new tserver with Master: c4fd17526c1c4a92a59c6d9bef4dd891 (127.17.109.195:33399)
I20260806 20:30:03.914929 5403 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.195:50689
I20260806 20:30:03.918598 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:30:03.928526 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.922785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33426 (local address 127.17.109.254:42355)
0806 20:30:03.922979 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:03.922983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:03.923464 (+ 481us) server_negotiation.cc:407] Connection header received
0806 20:30:03.923634 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:03.923638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:03.923706 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:03.923791 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:03.924245 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.924929 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.926342 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.926741 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.926836 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.927922 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:03.927927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:03.927930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:03.927946 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:03.927978 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:03.927989 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:03.928095 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:03.928277 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 20:30:03.928319 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 20:30:03.932698 5403 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:30:03.940729 5403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:30:03.974041 5606 heartbeater.cc:503] Master 127.17.109.254:42355 was elected leader, sending a full tablet report...
I20260806 20:30:03.981935 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.953397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39644 (local address 127.17.109.195:33399)
0806 20:30:03.969309 (+ 15912us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:03.969322 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:03.969370 (+ 48us) server_negotiation.cc:407] Connection header received
0806 20:30:03.969500 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:03.969512 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:03.969862 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:03.972464 (+ 2602us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:03.973698 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.975514 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.978957 (+ 3443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.980443 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.980669 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.980913 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:03.981419 (+ 506us) server_negotiation.cc:299] Negotiation successful
0806 20:30:03.981590 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15669,"thread_start_us":15478,"threads_started":1}
I20260806 20:30:03.981927 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.953162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45481 (local address 127.0.0.1:56844)
0806 20:30:03.971757 (+ 18595us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:03.971787 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:03.971931 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:03.973328 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:03.973333 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:03.973368 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:03.974581 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.974598 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.976676 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.976682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.978678 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.978691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.980968 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.980976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.981148 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.981182 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 20:30:03.981275 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 20:30:03.981483 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18322,"mutex_wait_us":291,"thread_start_us":19923,"threads_started":1}
I20260806 20:30:03.982237 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.953168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33399 (local address 127.0.0.1:39644)
0806 20:30:03.965212 (+ 12044us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:03.965252 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:03.965446 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:03.972803 (+ 7357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:03.972821 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:03.972851 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:03.973392 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.973409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.975736 (+ 2327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.975741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.978754 (+ 3013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.978770 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.980992 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.980999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.981208 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.981245 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 20:30:03.981352 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 20:30:03.981503 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11764,"mutex_wait_us":53,"thread_start_us":18183,"threads_started":1}
I20260806 20:30:03.982820 5915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.953702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:45167 (local address 127.0.0.1:57768)
0806 20:30:03.973664 (+ 19962us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:03.973678 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:03.973825 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:03.974698 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:03.974702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:03.974728 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:03.975085 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.975096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.976936 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.976941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.979433 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:03.979445 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.981949 (+ 2504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.981957 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:03.982179 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.982213 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 20:30:03.982312 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 20:30:03.982454 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19823,"mutex_wait_us":57,"thread_start_us":11087,"threads_started":1}
I20260806 20:30:03.982894 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.953923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.17.109.194:45481)
0806 20:30:03.972066 (+ 18143us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:03.972077 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:03.972109 (+ 32us) server_negotiation.cc:407] Connection header received
0806 20:30:03.972221 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:03.972230 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:03.972554 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:03.972804 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:03.974813 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.976490 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.978862 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.980536 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.980707 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.981926 (+ 1219us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:03.981998 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 20:30:03.982496 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17954,"thread_start_us":18095,"threads_started":1}
I20260806 20:30:03.983764 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:03.954301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.17.109.193:45167)
0806 20:30:03.972121 (+ 17820us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:03.972129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:03.973794 (+ 1665us) server_negotiation.cc:407] Connection header received
0806 20:30:03.974005 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:03.974011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:03.974182 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:03.974386 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:03.975271 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.976751 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.979641 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:03.981243 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:03.981450 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:03.981677 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:03.983163 (+ 1486us) server_negotiation.cc:299] Negotiation successful
0806 20:30:03.983349 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17628,"thread_start_us":17804,"threads_started":1}
I20260806 20:30:03.986771 5823 tablet_service.cc:1467] Processing CreateTablet for tablet af7f684b871a419ca0b3f450dfc0bda0 (DEFAULT_TABLE table=test-table [id=0808c7a839c248d49b1891e8d9cddc19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:03.986763 5680 tablet_service.cc:1467] Processing CreateTablet for tablet af7f684b871a419ca0b3f450dfc0bda0 (DEFAULT_TABLE table=test-table [id=0808c7a839c248d49b1891e8d9cddc19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:03.987277 5537 tablet_service.cc:1467] Processing CreateTablet for tablet af7f684b871a419ca0b3f450dfc0bda0 (DEFAULT_TABLE table=test-table [id=0808c7a839c248d49b1891e8d9cddc19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:03.988251 5680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af7f684b871a419ca0b3f450dfc0bda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:03.988335 5823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af7f684b871a419ca0b3f450dfc0bda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:03.988667 5537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af7f684b871a419ca0b3f450dfc0bda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:04.001245 5919 tablet_bootstrap.cc:492] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Bootstrap starting.
I20260806 20:30:04.003486 5921 tablet_bootstrap.cc:492] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Bootstrap starting.
I20260806 20:30:04.004527 5920 tablet_bootstrap.cc:492] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Bootstrap starting.
I20260806 20:30:04.005923 5919 tablet_bootstrap.cc:654] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:04.007718 5919 log.cc:824] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:04.008572 5920 tablet_bootstrap.cc:654] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:04.008944 5921 tablet_bootstrap.cc:654] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:04.009930 5920 log.cc:824] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:04.010318 5921 log.cc:824] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:04.010541 5919 tablet_bootstrap.cc:492] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: No bootstrap required, opened a new log
I20260806 20:30:04.010819 5919 ts_tablet_manager.cc:1397] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 20:30:04.013581 5921 tablet_bootstrap.cc:492] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: No bootstrap required, opened a new log
I20260806 20:30:04.013765 5920 tablet_bootstrap.cc:492] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: No bootstrap required, opened a new log
I20260806 20:30:04.013875 5921 ts_tablet_manager.cc:1397] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260806 20:30:04.014000 5920 ts_tablet_manager.cc:1397] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260806 20:30:04.021603 5921 raft_consensus.cc:348] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.021623 5919 raft_consensus.cc:348] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.022001 5921 raft_consensus.cc:374] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:04.022080 5919 raft_consensus.cc:374] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:04.022145 5921 raft_consensus.cc:729] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4fd17526c1c4a92a59c6d9bef4dd891, State: Initialized, Role: FOLLOWER
I20260806 20:30:04.022183 5919 raft_consensus.cc:729] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48d23afea9c64e80bd2f5623ff446a18, State: Initialized, Role: FOLLOWER
I20260806 20:30:04.022823 5921 consensus_queue.cc:260] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.022893 5919 consensus_queue.cc:260] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.024782 5921 ts_tablet_manager.cc:1428] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260806 20:30:04.024669 5920 raft_consensus.cc:348] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.025254 5920 raft_consensus.cc:374] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:04.025398 5920 raft_consensus.cc:729] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae884863896410ab3be9ff9c925f60b, State: Initialized, Role: FOLLOWER
I20260806 20:30:04.025411 5898 heartbeater.cc:503] Master 127.17.109.254:42355 was elected leader, sending a full tablet report...
I20260806 20:30:04.026054 5749 heartbeater.cc:503] Master 127.17.109.254:42355 was elected leader, sending a full tablet report...
I20260806 20:30:04.025986 5920 consensus_queue.cc:260] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.026644 5919 ts_tablet_manager.cc:1428] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260806 20:30:04.028589 5920 ts_tablet_manager.cc:1428] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260806 20:30:04.147544 5899 tablet.cc:2369] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:30:04.180052 5750 tablet.cc:2369] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:30:04.201046 5607 tablet.cc:2369] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:04.253005 5928 raft_consensus.cc:482] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:30:04.253459 5928 raft_consensus.cc:504] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.255699 5928 leader_election.cc:288] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4fd17526c1c4a92a59c6d9bef4dd891 (127.17.109.195:33399), 48d23afea9c64e80bd2f5623ff446a18 (127.17.109.194:45481)
I20260806 20:30:04.266304 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:04.256389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:48019 (local address 127.17.109.195:33399)
0806 20:30:04.256804 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:04.256813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:04.257328 (+ 515us) server_negotiation.cc:407] Connection header received
0806 20:30:04.257482 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:04.257489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:04.257582 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:04.257764 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:04.258639 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.259923 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.263549 (+ 3626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.265071 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.265238 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:04.265316 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:04.265850 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 20:30:04.266005 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247}
I20260806 20:30:04.266355 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:04.256309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:59213 (local address 127.17.109.194:45481)
0806 20:30:04.256691 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:04.256702 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:04.257208 (+ 506us) server_negotiation.cc:407] Connection header received
0806 20:30:04.257483 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:04.257489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:04.257582 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:04.257700 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:04.258639 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.259916 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.263549 (+ 3633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.265029 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.265236 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:04.265316 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:04.265850 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 20:30:04.266028 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260806 20:30:04.266831 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:04.255971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45481 (local address 127.17.109.193:59213)
0806 20:30:04.257066 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:04.257088 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:04.257299 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:04.257772 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:04.257778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:04.257810 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:04.258478 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:04.258489 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.260086 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.260092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:04.263272 (+ 3180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:04.263285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.265330 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.265336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:04.265597 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:04.265664 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 20:30:04.265774 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 20:30:04.265976 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":325,"thread_start_us":888,"threads_started":1}
I20260806 20:30:04.266861 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:04.256102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:33399 (local address 127.17.109.193:48019)
0806 20:30:04.257216 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:04.257234 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:04.257345 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:04.257772 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:04.257778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:04.257880 (+ 102us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:04.258400 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:04.258420 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.260088 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.260092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:04.263268 (+ 3176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:04.263285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.265300 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.265314 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:04.265592 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:04.265664 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 20:30:04.265774 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 20:30:04.265976 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":326,"spinlock_wait_cycles":23424,"thread_start_us":608,"threads_started":1}
I20260806 20:30:04.268193 5849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af7f684b871a419ca0b3f450dfc0bda0" candidate_uuid: "cae884863896410ab3be9ff9c925f60b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" is_pre_election: true
I20260806 20:30:04.268190 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af7f684b871a419ca0b3f450dfc0bda0" candidate_uuid: "cae884863896410ab3be9ff9c925f60b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48d23afea9c64e80bd2f5623ff446a18" is_pre_election: true
I20260806 20:30:04.268769 5849 raft_consensus.cc:2454] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae884863896410ab3be9ff9c925f60b in term 0.
I20260806 20:30:04.268803 5702 raft_consensus.cc:2454] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cae884863896410ab3be9ff9c925f60b in term 0.
I20260806 20:30:04.269567 5484 leader_election.cc:302] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 48d23afea9c64e80bd2f5623ff446a18, c4fd17526c1c4a92a59c6d9bef4dd891, cae884863896410ab3be9ff9c925f60b; no voters:
I20260806 20:30:04.270103 5928 raft_consensus.cc:2781] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:30:04.270313 5928 raft_consensus.cc:482] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:30:04.270473 5928 raft_consensus.cc:3037] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:04.272981 5928 raft_consensus.cc:504] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.273712 5928 leader_election.cc:288] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [CANDIDATE]: Term 1 election: Requested vote from peers c4fd17526c1c4a92a59c6d9bef4dd891 (127.17.109.195:33399), 48d23afea9c64e80bd2f5623ff446a18 (127.17.109.194:45481)
I20260806 20:30:04.274251 5702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af7f684b871a419ca0b3f450dfc0bda0" candidate_uuid: "cae884863896410ab3be9ff9c925f60b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48d23afea9c64e80bd2f5623ff446a18"
I20260806 20:30:04.274263 5849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af7f684b871a419ca0b3f450dfc0bda0" candidate_uuid: "cae884863896410ab3be9ff9c925f60b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891"
I20260806 20:30:04.274482 5849 raft_consensus.cc:3037] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:04.274487 5702 raft_consensus.cc:3037] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:04.277000 5849 raft_consensus.cc:2454] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae884863896410ab3be9ff9c925f60b in term 1.
I20260806 20:30:04.277019 5702 raft_consensus.cc:2454] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cae884863896410ab3be9ff9c925f60b in term 1.
I20260806 20:30:04.277473 5486 leader_election.cc:302] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4fd17526c1c4a92a59c6d9bef4dd891, cae884863896410ab3be9ff9c925f60b; no voters:
I20260806 20:30:04.277855 5928 raft_consensus.cc:2781] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:04.278337 5928 raft_consensus.cc:686] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 1 LEADER]: Becoming Leader. State: Replica: cae884863896410ab3be9ff9c925f60b, State: Running, Role: LEADER
I20260806 20:30:04.278874 5928 consensus_queue.cc:237] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } }
I20260806 20:30:04.283773 5403 catalog_manager.cc:5495] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b reported cstate change: term changed from 0 to 1, leader changed from <none> to cae884863896410ab3be9ff9c925f60b (127.17.109.193). New cstate: current_term: 1 leader_uuid: "cae884863896410ab3be9ff9c925f60b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cae884863896410ab3be9ff9c925f60b" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 45167 } health_report { overall_health: HEALTHY } } }
I20260806 20:30:04.354677 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:04.349115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57784 (local address 127.17.109.193:45167)
0806 20:30:04.349377 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:04.349381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:04.349409 (+ 28us) server_negotiation.cc:407] Connection header received
0806 20:30:04.349609 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:04.349613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:04.349706 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:04.349839 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:04.350206 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.350902 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.352450 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.352886 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.352980 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:04.353511 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:04.354001 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:04.354132 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:04.354335 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 20:30:04.354384 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 20:30:04.363848 5702 raft_consensus.cc:1261] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 1 FOLLOWER]: Refusing update from remote peer cae884863896410ab3be9ff9c925f60b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:04.363852 5849 raft_consensus.cc:1261] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 1 FOLLOWER]: Refusing update from remote peer cae884863896410ab3be9ff9c925f60b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:04.364921 5931 consensus_queue.cc:1035] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4fd17526c1c4a92a59c6d9bef4dd891" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 33399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:04.365311 5928 consensus_queue.cc:1035] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [LEADER]: Connected to new peer: Peer: permanent_uuid: "48d23afea9c64e80bd2f5623ff446a18" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:04.380687 5935 mvcc.cc:205] Tried to move back new op lower bound from 7315653445062602752 to 7315653444730634240. Current Snapshot: MvccSnapshot[applied={T|T < 7315653445062602752}]
I20260806 20:30:04.399802 5401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:30:04.400162 5401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:30:04.404834 5401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f9396649bc24ffea5556621be0eb258: Sending DeleteTablet for 3 replicas of tablet af7f684b871a419ca0b3f450dfc0bda0
I20260806 20:30:04.406090 5823 tablet_service.cc:1514] Processing DeleteTablet for tablet af7f684b871a419ca0b3f450dfc0bda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:04 UTC) from {username='slave'} at 127.0.0.1:39644
I20260806 20:30:04.406852 5680 tablet_service.cc:1514] Processing DeleteTablet for tablet af7f684b871a419ca0b3f450dfc0bda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:04 UTC) from {username='slave'} at 127.0.0.1:56844
I20260806 20:30:04.408620 5537 tablet_service.cc:1514] Processing DeleteTablet for tablet af7f684b871a419ca0b3f450dfc0bda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:04 UTC) from {username='slave'} at 127.0.0.1:57768
I20260806 20:30:04.410324 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 5468
I20260806 20:30:04.420168 5945 tablet_replica.cc:330] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: stopping tablet replica
I20260806 20:30:04.420570 5946 tablet_replica.cc:330] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: stopping tablet replica
I20260806 20:30:04.420804 5945 raft_consensus.cc:2229] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:30:04.421200 5946 raft_consensus.cc:2229] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:30:04.421483 5945 raft_consensus.cc:2258] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:04.421875 5946 raft_consensus.cc:2258] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:04.422778 5947 tablet_replica.cc:330] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: stopping tablet replica
I20260806 20:30:04.423310 5947 raft_consensus.cc:2229] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:30:04.424165 5947 raft_consensus.cc:2258] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:04.424546 5946 ts_tablet_manager.cc:1905] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:04.426682 5947 ts_tablet_manager.cc:1905] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:04.435927 5947 ts_tablet_manager.cc:1918] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:04.436247 5947 log.cc:1196] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/af7f684b871a419ca0b3f450dfc0bda0
I20260806 20:30:04.436800 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:04.412987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.17.109.193:45167)
0806 20:30:04.413246 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:04.413256 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:04.418791 (+ 5535us) server_negotiation.cc:407] Connection header received
0806 20:30:04.422034 (+ 3243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:04.422045 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:04.422223 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:04.422323 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:04.423103 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.424581 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.426778 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.427608 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.427729 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:04.434690 (+ 6961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:04.434704 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:04.434723 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:04.435628 (+ 905us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:04.435696 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:04.435723 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:04.435836 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:04.436237 (+ 401us) server_negotiation.cc:299] Negotiation successful
0806 20:30:04.436376 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:30:04.437251 5947 ts_tablet_manager.cc:1939] T af7f684b871a419ca0b3f450dfc0bda0 P cae884863896410ab3be9ff9c925f60b: Deleting consensus metadata
I20260806 20:30:04.438166 5601 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:04.445719 5945 ts_tablet_manager.cc:1905] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:04.448581 5946 ts_tablet_manager.cc:1918] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:04.448938 5946 log.cc:1196] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/af7f684b871a419ca0b3f450dfc0bda0
I20260806 20:30:04.449779 5946 ts_tablet_manager.cc:1939] T af7f684b871a419ca0b3f450dfc0bda0 P 48d23afea9c64e80bd2f5623ff446a18: Deleting consensus metadata
I20260806 20:30:04.452416 5384 catalog_manager.cc:4841] TS 48d23afea9c64e80bd2f5623ff446a18 (127.17.109.194:45481): tablet af7f684b871a419ca0b3f450dfc0bda0 (table test-table [id=0808c7a839c248d49b1891e8d9cddc19]) successfully deleted
I20260806 20:30:04.454276 5945 ts_tablet_manager.cc:1918] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:04.454481 5945 log.cc:1196] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/af7f684b871a419ca0b3f450dfc0bda0
I20260806 20:30:04.455116 5945 ts_tablet_manager.cc:1939] T af7f684b871a419ca0b3f450dfc0bda0 P c4fd17526c1c4a92a59c6d9bef4dd891: Deleting consensus metadata
I20260806 20:30:04.457129 5386 catalog_manager.cc:4841] TS c4fd17526c1c4a92a59c6d9bef4dd891 (127.17.109.195:33399): tablet af7f684b871a419ca0b3f450dfc0bda0 (table test-table [id=0808c7a839c248d49b1891e8d9cddc19]) successfully deleted
I20260806 20:30:04.552856 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 5468
I20260806 20:30:04.552997 5385 catalog_manager.cc:4841] TS cae884863896410ab3be9ff9c925f60b (127.17.109.193:45167): tablet af7f684b871a419ca0b3f450dfc0bda0 (table test-table [id=0808c7a839c248d49b1891e8d9cddc19]) successfully deleted
I20260806 20:30:04.573524 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 5610
I20260806 20:30:04.581022 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:04.574386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.17.109.194:45481)
0806 20:30:04.574594 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:04.574603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:04.574682 (+ 79us) server_negotiation.cc:407] Connection header received
0806 20:30:04.574899 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:04.574911 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:04.574998 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:04.575117 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:04.575535 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.576352 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.577677 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.578112 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.578272 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:04.579438 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:04.579447 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:04.579461 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:04.580214 (+ 753us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:04.580263 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:04.580310 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:04.580437 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:04.580571 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 20:30:04.580680 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 20:30:04.581755 5744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:04.673568 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 5610
I20260806 20:30:04.694311 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 5752
I20260806 20:30:04.702802 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:04.695261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39650 (local address 127.17.109.195:33399)
0806 20:30:04.695476 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:04.695482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:04.695506 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:30:04.695787 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:04.695794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:04.695899 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:04.696021 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:04.696475 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.697273 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.699075 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:04.699826 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:04.699974 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:04.701229 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:04.701247 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:04.701263 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:04.701868 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:04.701919 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:04.701944 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:04.702090 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:04.702215 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 20:30:04.702334 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:30:04.703608 5893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:04.799858 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 5752
I20260806 20:30:04.819708 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 5368
I20260806 20:30:04.820912 5435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:04.934057 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 5368
2026-08-06T20:30:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3085 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:05 dist-test-slave-6k22 krb5kdc[5956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:05 dist-test-slave-6k22 krb5kdc[5956](info): set up 2 sockets
Aug 06 20:30:05 dist-test-slave-6k22 krb5kdc[5956](info): commencing operation
krb5kdc: starting...
W20260806 20:30:06.178431 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.003s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:06 dist-test-slave-6k22 krb5kdc[5956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:06.503399 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34529
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:06.846602 5971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:06.846908 5971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:06.846999 5971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:06.854318 5971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:06.854429 5971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:06.854477 5971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:06.854544 5971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:06.854589 5971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:06.864228 5971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:06.865458 5971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:06.866735 5971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:06.880177 5976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:06.880082 5977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:06.880498 5979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:06.881346 5971 server_base.cc:1034] running on GCE node
I20260806 20:30:06.882063 5971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:06.883411 5971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:06.884625 5971 hybrid_clock.cc:648] HybridClock initialized: now 1786048206884565 us; error 51 us; skew 500 ppm
I20260806 20:30:06.885036 5971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:06 dist-test-slave-6k22 krb5kdc[5956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048206, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:06.894025 5971 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:06.895696 5971 webserver.cc:460] Webserver started at http://127.17.109.254:44505/ using document root <none> and password file <none>
I20260806 20:30:06.896412 5971 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:06.896544 5971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:06.896875 5971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:06.900362 5971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908"
format_stamp: "Formatted at 2026-08-06 20:30:06 on dist-test-slave-6k22"
server_key: "790e6cfcb6facfc8b5e51e8edebea867"
server_key_iv: "6fae720e5c4168698715d5db64f33994"
server_key_version: "encryptionkey@0"
I20260806 20:30:06.901121 5971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908"
format_stamp: "Formatted at 2026-08-06 20:30:06 on dist-test-slave-6k22"
server_key: "790e6cfcb6facfc8b5e51e8edebea867"
server_key_iv: "6fae720e5c4168698715d5db64f33994"
server_key_version: "encryptionkey@0"
I20260806 20:30:06.906337 5971 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 20:30:06.910179 5986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:06.911669 5971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:30:06.911861 5971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908"
format_stamp: "Formatted at 2026-08-06 20:30:06 on dist-test-slave-6k22"
server_key: "790e6cfcb6facfc8b5e51e8edebea867"
server_key_iv: "6fae720e5c4168698715d5db64f33994"
server_key_version: "encryptionkey@0"
I20260806 20:30:06.912082 5971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:06.938393 5971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:06.942767 5971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:06.943075 5971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:06.962514 5971 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:34529
I20260806 20:30:06.962538 6040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:34529 every 8 connection(s)
I20260806 20:30:06.964037 5971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:06.965750 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 5971
I20260806 20:30:06.965936 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:06.966516 17847 external_mini_cluster.cc:1428] Setting key 532446d69cd0e5e29fcf34a4f494824d
I20260806 20:30:06.968781 6041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 20:30:06 dist-test-slave-6k22 krb5kdc[5956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:06.982895 6041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908: Bootstrap starting.
I20260806 20:30:06.986253 6041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:06.987326 6041 log.cc:824] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:06.990469 6041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908: No bootstrap required, opened a new log
I20260806 20:30:06.990723 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:06.968461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.17.109.254:34529)
0806 20:30:06.969274 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:06.969304 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:06.969345 (+ 41us) server_negotiation.cc:407] Connection header received
0806 20:30:06.971021 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:06.971043 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:06.971439 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:06.971881 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:06.973824 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:06.975174 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:06.977095 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:06.977832 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:06.977970 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:06.983162 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:06.983172 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:06.983188 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:06.983230 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:06.986445 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:06.987208 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:06.987215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:06.987221 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:06.987400 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:06.987855 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:06.987859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:06.987861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:06.988429 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:06.988763 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:06.989245 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 20:30:06.989682 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":382,"threads_started":1}
I20260806 20:30:06.998723 6041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34529 } }
I20260806 20:30:06.999130 6041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:06.999238 6041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27f4563ddcbd4bf0a1bb54e7adcdd908, State: Initialized, Role: FOLLOWER
I20260806 20:30:06.999840 6041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34529 } }
I20260806 20:30:07.000098 6041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:07.000260 6041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:07.000424 6041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:07.002600 6041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34529 } }
I20260806 20:30:07.003124 6041 leader_election.cc:302] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27f4563ddcbd4bf0a1bb54e7adcdd908; no voters:
I20260806 20:30:07.003819 6041 leader_election.cc:288] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:07.003952 6047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:07.005230 6047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [term 1 LEADER]: Becoming Leader. State: Replica: 27f4563ddcbd4bf0a1bb54e7adcdd908, State: Running, Role: LEADER
I20260806 20:30:07.005878 6047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34529 } }
I20260806 20:30:07.006035 6041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:07.009140 6049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27f4563ddcbd4bf0a1bb54e7adcdd908. Latest consensus state: current_term: 1 leader_uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34529 } } }
I20260806 20:30:07.009215 6048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27f4563ddcbd4bf0a1bb54e7adcdd908" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34529 } } }
I20260806 20:30:07.009512 6049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:07.009570 6048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:07.010346 6056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:07.019114 6056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:07.030134 6056 catalog_manager.cc:1347] Generated new cluster ID: b9b50613fa2348d89cd7cd6a3136bec2
I20260806 20:30:07.030282 6056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:07.046768 6056 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:07.048682 6056 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:07.059363 6056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908: Generated new TSK 0
I20260806 20:30:07.060289 6056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.109.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
Entry for principal kudu/127.17.109.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab.
I20260806 20:30:07.146633 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34529
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:07.490718 6068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:07.491006 6068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:07.491163 6068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:07.498795 6068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:07.498920 6068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:07.499089 6068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:30:07.509356 6068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.193.keytab
--principal=kudu/127.17.109.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:07.510847 6068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:07.512301 6068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:07.525573 6078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:07.525489 6075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:07.525520 6074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:07.526461 6068 server_base.cc:1034] running on GCE node
I20260806 20:30:07.527000 6068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:07.528023 6068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:07.529256 6068 hybrid_clock.cc:648] HybridClock initialized: now 1786048207529197 us; error 81 us; skew 500 ppm
I20260806 20:30:07.529587 6068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:07 dist-test-slave-6k22 krb5kdc[5956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048207, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:07.533044 6068 init.cc:377] Logged in from keytab as kudu/127.17.109.193@KRBTEST.COM (short username kudu)
I20260806 20:30:07.534564 6068 webserver.cc:460] Webserver started at http://127.17.109.193:43009/ using document root <none> and password file <none>
I20260806 20:30:07.535264 6068 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:07.535421 6068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:07.535866 6068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:07.538782 6068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "dc0cd971edad49a6a13f89a28077325d"
format_stamp: "Formatted at 2026-08-06 20:30:07 on dist-test-slave-6k22"
server_key: "aa7b84a37913ce96f9c9501a14ef9edb"
server_key_iv: "d517c08cacd0125a0db1db3bda995a0e"
server_key_version: "encryptionkey@0"
I20260806 20:30:07.539556 6068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "dc0cd971edad49a6a13f89a28077325d"
format_stamp: "Formatted at 2026-08-06 20:30:07 on dist-test-slave-6k22"
server_key: "aa7b84a37913ce96f9c9501a14ef9edb"
server_key_iv: "d517c08cacd0125a0db1db3bda995a0e"
server_key_version: "encryptionkey@0"
I20260806 20:30:07.544801 6068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:30:07.548465 6086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:07.549772 6068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:07.549952 6068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "dc0cd971edad49a6a13f89a28077325d"
format_stamp: "Formatted at 2026-08-06 20:30:07 on dist-test-slave-6k22"
server_key: "aa7b84a37913ce96f9c9501a14ef9edb"
server_key_iv: "d517c08cacd0125a0db1db3bda995a0e"
server_key_version: "encryptionkey@0"
I20260806 20:30:07.550171 6068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:07.571967 6068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:07.576442 6068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:07.576752 6068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:07.577776 6068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:07.579211 6068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:07.579291 6068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:07.579438 6068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:07.579511 6068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:07.615324 6068 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:38097
I20260806 20:30:07.615440 6208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:38097 every 8 connection(s)
I20260806 20:30:07.616771 6068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:30:07.617437 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 6068
I20260806 20:30:07.617678 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:30:07.618398 17847 external_mini_cluster.cc:1428] Setting key 8051ae895339e4bcd3e37a303ec5b4f1
Aug 06 20:30:07 dist-test-slave-6k22 krb5kdc[5956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048207, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.193@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:07.639392 6211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:07.619650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:48485 (local address 127.17.109.254:34529)
0806 20:30:07.620242 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:07.620248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:07.621289 (+ 1041us) server_negotiation.cc:407] Connection header received
0806 20:30:07.623299 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:07.623305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:07.623399 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:07.623533 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:07.625155 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:07.625952 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:07.627624 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:07.628085 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:07.628247 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:07.632005 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:07.632010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:07.632014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:07.632055 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:07.635855 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:07.636600 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:07.636609 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:07.636617 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:07.636790 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:07.637228 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:07.637237 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:07.637243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:07.637680 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:07.637851 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:07.638561 (+ 710us) server_negotiation.cc:299] Negotiation successful
0806 20:30:07.638861 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":248,"threads_started":1}
I20260806 20:30:07.639928 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:07.620234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34529 (local address 127.17.109.193:48485)
0806 20:30:07.621109 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:07.621148 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:07.623011 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:07.623809 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:07.623818 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:07.624372 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:07.624953 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:07.624971 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:07.626109 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:07.626115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:07.627451 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:07.627464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:07.628346 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:07.628356 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:07.628608 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:07.629350 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:07.629379 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:07.631774 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:07.635998 (+ 4224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:07.636005 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:07.636016 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:07.636462 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:07.636905 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:07.636909 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:07.636911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:07.637088 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:07.637810 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:07.637816 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:07.638124 (+ 308us) client_negotiation.cc:769] Sending connection context
0806 20:30:07.638361 (+ 237us) client_negotiation.cc:240] Negotiation successful
0806 20:30:07.638757 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":383,"threads_started":1}
I20260806 20:30:07.641577 6209 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34529
I20260806 20:30:07.641884 6209 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:07.642598 6209 heartbeater.cc:511] Master 127.17.109.254:34529 requested a full tablet report, sending...
I20260806 20:30:07.644678 6004 ts_manager.cc:194] Registered new tserver with Master: dc0cd971edad49a6a13f89a28077325d (127.17.109.193:38097)
I20260806 20:30:07.646132 6004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.193@KRBTEST.COM'} at 127.17.109.193:48485
WARNING: no policy specified for kudu/127.17.109.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
Entry for principal kudu/127.17.109.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab.
I20260806 20:30:07.691944 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34529
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:08.030337 6215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:08.030627 6215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:08.030799 6215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:08.038259 6215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:08.038401 6215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:08.038606 6215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:30:08.048619 6215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.194.keytab
--principal=kudu/127.17.109.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:08.049983 6215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:08.051362 6215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:08.066272 6222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:08.066442 6223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:08.066234 6225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:08.067139 6215 server_base.cc:1034] running on GCE node
I20260806 20:30:08.067866 6215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:08.068848 6215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:08.070093 6215 hybrid_clock.cc:648] HybridClock initialized: now 1786048208070046 us; error 44 us; skew 500 ppm
I20260806 20:30:08.070446 6215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:08 dist-test-slave-6k22 krb5kdc[5956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048208, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:08.073835 6215 init.cc:377] Logged in from keytab as kudu/127.17.109.194@KRBTEST.COM (short username kudu)
I20260806 20:30:08.075425 6215 webserver.cc:460] Webserver started at http://127.17.109.194:44845/ using document root <none> and password file <none>
I20260806 20:30:08.076175 6215 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:08.076327 6215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:08.076653 6215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:08.079680 6215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "951661b8953442c4846885cb17753c0e"
format_stamp: "Formatted at 2026-08-06 20:30:08 on dist-test-slave-6k22"
server_key: "f252d76444bf98b5bc8aac12aa5dbf76"
server_key_iv: "d718bbc19912415fd345a523a78a0173"
server_key_version: "encryptionkey@0"
I20260806 20:30:08.080669 6215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "951661b8953442c4846885cb17753c0e"
format_stamp: "Formatted at 2026-08-06 20:30:08 on dist-test-slave-6k22"
server_key: "f252d76444bf98b5bc8aac12aa5dbf76"
server_key_iv: "d718bbc19912415fd345a523a78a0173"
server_key_version: "encryptionkey@0"
I20260806 20:30:08.086242 6215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:30:08.090065 6232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:08.091490 6215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:30:08.091741 6215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "951661b8953442c4846885cb17753c0e"
format_stamp: "Formatted at 2026-08-06 20:30:08 on dist-test-slave-6k22"
server_key: "f252d76444bf98b5bc8aac12aa5dbf76"
server_key_iv: "d718bbc19912415fd345a523a78a0173"
server_key_version: "encryptionkey@0"
I20260806 20:30:08.092000 6215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:08.108706 6215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:08.113096 6215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:08.113396 6215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:08.114413 6215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:08.115919 6215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:08.116021 6215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:08.116176 6215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:08.116277 6215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:08.150736 6215 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:44963
I20260806 20:30:08.150838 6354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:44963 every 8 connection(s)
I20260806 20:30:08.152217 6215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:30:08.154520 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 6215
I20260806 20:30:08.154738 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:30:08.155380 17847 external_mini_cluster.cc:1428] Setting key d878fd4e6e95b29f96a086388077955c
Aug 06 20:30:08 dist-test-slave-6k22 krb5kdc[5956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048208, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.194@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:08.176195 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.155217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:45211 (local address 127.17.109.254:34529)
0806 20:30:08.155771 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:08.155778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:08.157133 (+ 1355us) server_negotiation.cc:407] Connection header received
0806 20:30:08.159275 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:08.159280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:08.159365 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:08.159496 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:08.161142 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.161932 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.163534 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.164012 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.164156 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.168131 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:08.168136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:08.168140 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:08.168174 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:08.171429 (+ 3255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:08.172441 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:08.172449 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:08.172455 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:08.172618 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:08.173291 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:08.173321 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:08.173327 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:08.173715 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:08.173871 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:08.175397 (+ 1526us) server_negotiation.cc:299] Negotiation successful
0806 20:30:08.175703 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":259,"threads_started":1}
I20260806 20:30:08.178413 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.155994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34529 (local address 127.17.109.194:45211)
0806 20:30:08.156928 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:08.156981 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:08.158975 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:08.159791 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:08.159801 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:08.160344 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:08.160956 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.160975 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.162113 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.162117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.163380 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.163392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.164251 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.164260 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.164513 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.165246 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:08.165270 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:08.167875 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:08.171743 (+ 3868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:08.171751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:08.171764 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:08.172264 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:08.172778 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:08.172805 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:08.172811 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:08.173143 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:08.173876 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:08.173888 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:08.174723 (+ 835us) client_negotiation.cc:769] Sending connection context
0806 20:30:08.175273 (+ 550us) client_negotiation.cc:240] Negotiation successful
0806 20:30:08.176298 (+ 1025us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":379,"threads_started":1}
I20260806 20:30:08.181653 6355 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34529
I20260806 20:30:08.182018 6355 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:08.182901 6355 heartbeater.cc:511] Master 127.17.109.254:34529 requested a full tablet report, sending...
I20260806 20:30:08.184692 6004 ts_manager.cc:194] Registered new tserver with Master: 951661b8953442c4846885cb17753c0e (127.17.109.194:44963)
I20260806 20:30:08.185631 6004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.194@KRBTEST.COM'} at 127.17.109.194:45211
WARNING: no policy specified for kudu/127.17.109.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
Entry for principal kudu/127.17.109.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab.
I20260806 20:30:08.231662 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34529
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:08.578662 6362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:08.578975 6362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:08.579149 6362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:08.586935 6362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:08.587109 6362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:08.587309 6362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:30:08.599352 6362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.195.keytab
--principal=kudu/127.17.109.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:34529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:08.600831 6362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:08.602283 6362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:08.617360 6368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:08.617483 6367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:08.618089 6370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:08.618403 6362 server_base.cc:1034] running on GCE node
I20260806 20:30:08.619074 6362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:08.620252 6362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:08.621556 6362 hybrid_clock.cc:648] HybridClock initialized: now 1786048208621506 us; error 44 us; skew 500 ppm
I20260806 20:30:08.621964 6362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:08 dist-test-slave-6k22 krb5kdc[5956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048208, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:08.625257 6362 init.cc:377] Logged in from keytab as kudu/127.17.109.195@KRBTEST.COM (short username kudu)
I20260806 20:30:08.626936 6362 webserver.cc:460] Webserver started at http://127.17.109.195:43835/ using document root <none> and password file <none>
I20260806 20:30:08.627758 6362 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:08.627918 6362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:08.628242 6362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:08.631197 6362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d"
format_stamp: "Formatted at 2026-08-06 20:30:08 on dist-test-slave-6k22"
server_key: "097031a99397279fa21d0a1ef1dcd93f"
server_key_iv: "b55d59d691c6d7e7375037c44d30c5a4"
server_key_version: "encryptionkey@0"
I20260806 20:30:08.632035 6362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d"
format_stamp: "Formatted at 2026-08-06 20:30:08 on dist-test-slave-6k22"
server_key: "097031a99397279fa21d0a1ef1dcd93f"
server_key_iv: "b55d59d691c6d7e7375037c44d30c5a4"
server_key_version: "encryptionkey@0"
I20260806 20:30:08.637253 6362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:30:08.640668 6380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:08.642055 6362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:30:08.642237 6362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d"
format_stamp: "Formatted at 2026-08-06 20:30:08 on dist-test-slave-6k22"
server_key: "097031a99397279fa21d0a1ef1dcd93f"
server_key_iv: "b55d59d691c6d7e7375037c44d30c5a4"
server_key_version: "encryptionkey@0"
I20260806 20:30:08.642436 6362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:08.649052 6209 heartbeater.cc:503] Master 127.17.109.254:34529 was elected leader, sending a full tablet report...
I20260806 20:30:08.659075 6362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:08.663478 6362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:08.663811 6362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:08.664734 6362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:08.665993 6362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:08.666074 6362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:08.666164 6362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:08.666209 6362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:08.700954 6362 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:43001
I20260806 20:30:08.700999 6507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:43001 every 8 connection(s)
I20260806 20:30:08.702347 6362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:30:08.703686 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 6362
I20260806 20:30:08.703893 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:30:08.704511 17847 external_mini_cluster.cc:1428] Setting key 235a1b83b9bd0db588372034dbf6f315
Aug 06 20:30:08 dist-test-slave-6k22 krb5kdc[5956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048208, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.195@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:08.723296 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.705641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:57797 (local address 127.17.109.254:34529)
0806 20:30:08.706115 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:08.706122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:08.707343 (+ 1221us) server_negotiation.cc:407] Connection header received
0806 20:30:08.709319 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:08.709323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:08.709401 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:08.709491 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:08.711181 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.712051 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.713515 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.713894 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.713984 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.717436 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:08.717441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:08.717444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:08.717486 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:08.720352 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:08.721087 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:08.721094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:08.721097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:08.721226 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:08.721666 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:08.721672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:08.721675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:08.722003 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:08.722168 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:08.722825 (+ 657us) server_negotiation.cc:299] Negotiation successful
0806 20:30:08.722993 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":237,"threads_started":1}
I20260806 20:30:08.724332 6513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.706225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34529 (local address 127.17.109.195:57797)
0806 20:30:08.707159 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:08.707223 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:08.709072 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:08.709702 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:08.709710 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:08.710340 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:08.711023 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.711055 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.712185 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.712189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.713393 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.713404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.714079 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.714083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.714275 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.714898 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:08.714919 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:08.717257 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:08.720500 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:08.720509 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:08.720526 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:08.720963 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:08.721351 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:08.721356 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:08.721360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:08.721549 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:08.722143 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:08.722150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:08.722473 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 20:30:08.722715 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 20:30:08.723115 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":371,"threads_started":1}
I20260806 20:30:08.725777 6509 heartbeater.cc:347] Connected to a master server at 127.17.109.254:34529
I20260806 20:30:08.726104 6509 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:08.726859 6509 heartbeater.cc:511] Master 127.17.109.254:34529 requested a full tablet report, sending...
I20260806 20:30:08.728363 6004 ts_manager.cc:194] Registered new tserver with Master: 5a5fbc8eb4dd4b7880a4403cbdc54f4d (127.17.109.195:43001)
I20260806 20:30:08.729197 6004 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.109.195@KRBTEST.COM'} at 127.17.109.195:57797
I20260806 20:30:08.732820 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:30:08.750710 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.737564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.17.109.254:34529)
0806 20:30:08.737760 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:08.737766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:08.738158 (+ 392us) server_negotiation.cc:407] Connection header received
0806 20:30:08.738363 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:08.738379 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:08.738454 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:08.738559 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:08.740085 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.740820 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.742131 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.742537 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.742673 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.745094 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:08.745107 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:08.745110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:08.745143 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:08.748018 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:08.748681 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:08.748688 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:08.748691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:08.748819 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:08.749208 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:08.749216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:08.749219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:08.749547 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:08.749802 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:08.750166 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 20:30:08.750399 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 20:30:08.754830 6004 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:30:08.761831 6004 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:30:08.787343 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.775534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57996 (local address 127.17.109.195:43001)
0806 20:30:08.777602 (+ 2068us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:08.777625 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:08.779039 (+ 1414us) server_negotiation.cc:407] Connection header received
0806 20:30:08.779164 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:08.779172 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:08.779400 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:08.779620 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:08.780768 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.782407 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.784496 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.785925 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.786120 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.786326 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:08.786823 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 20:30:08.786991 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1879,"thread_start_us":1692,"threads_started":1}
I20260806 20:30:08.787418 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.775067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50368 (local address 127.17.109.193:38097)
0806 20:30:08.778165 (+ 3098us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:08.778176 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:08.778210 (+ 34us) server_negotiation.cc:407] Connection header received
0806 20:30:08.778903 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:08.778914 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:08.779191 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:08.779414 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:08.780396 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.781846 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.784795 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.786214 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.786414 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.786584 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:08.786936 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 20:30:08.787086 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2917,"thread_start_us":2728,"threads_started":1}
I20260806 20:30:08.787423 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.774837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:43001 (local address 127.0.0.1:57996)
0806 20:30:08.778704 (+ 3867us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:08.778777 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:08.778902 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:08.779899 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:08.779903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:08.779925 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:08.780542 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.780561 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.782559 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.782563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.784342 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.784355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.786322 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.786329 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.786548 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.786600 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 20:30:08.786732 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 20:30:08.786957 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3729,"mutex_wait_us":2,"thread_start_us":777,"threads_started":1}
I20260806 20:30:08.788149 6524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.774191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:38097 (local address 127.0.0.1:50368)
0806 20:30:08.777189 (+ 2998us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:08.777216 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:08.778640 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:08.779414 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:08.779424 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:08.779459 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:08.780057 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.780074 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.782013 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.782019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.784610 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.784625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.786486 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.786493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.786658 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.786688 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 20:30:08.786794 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 20:30:08.787734 (+ 940us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2848,"mutex_wait_us":798,"thread_start_us":4216,"threads_started":1}
I20260806 20:30:08.788297 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.775249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46136 (local address 127.17.109.194:44963)
0806 20:30:08.779094 (+ 3845us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:08.779105 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:08.779127 (+ 22us) server_negotiation.cc:407] Connection header received
0806 20:30:08.779208 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:08.779213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:08.779399 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:08.779620 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:08.780691 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.782335 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.784998 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.786375 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.786541 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.786701 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:08.787828 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0806 20:30:08.788007 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3638,"thread_start_us":3542,"threads_started":1}
I20260806 20:30:08.788257 6526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:08.774135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44963 (local address 127.0.0.1:46136)
0806 20:30:08.776194 (+ 2059us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:08.776268 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:08.776504 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:08.779747 (+ 3243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:08.779752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:08.779777 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:08.780512 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.780526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.782484 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.782491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.784848 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:08.784858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:08.786863 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:08.786869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:08.787033 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:08.787072 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 20:30:08.787154 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 20:30:08.787276 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1692,"mutex_wait_us":3,"thread_start_us":2723,"threads_started":1}
I20260806 20:30:08.791711 6433 tablet_service.cc:1467] Processing CreateTablet for tablet 65908f5b0c6f420e9a9193ed5f457f4b (DEFAULT_TABLE table=test-table [id=a8b3a968258d48659df624e25f8c455a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:08.791945 6285 tablet_service.cc:1467] Processing CreateTablet for tablet 65908f5b0c6f420e9a9193ed5f457f4b (DEFAULT_TABLE table=test-table [id=a8b3a968258d48659df624e25f8c455a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:08.791996 6138 tablet_service.cc:1467] Processing CreateTablet for tablet 65908f5b0c6f420e9a9193ed5f457f4b (DEFAULT_TABLE table=test-table [id=a8b3a968258d48659df624e25f8c455a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:08.793118 6285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65908f5b0c6f420e9a9193ed5f457f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:08.793278 6433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65908f5b0c6f420e9a9193ed5f457f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:08.793298 6138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65908f5b0c6f420e9a9193ed5f457f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:08.805390 6531 tablet_bootstrap.cc:492] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Bootstrap starting.
I20260806 20:30:08.808070 6533 tablet_bootstrap.cc:492] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Bootstrap starting.
I20260806 20:30:08.809592 6531 tablet_bootstrap.cc:654] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:08.810981 6531 log.cc:824] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:08.812081 6533 tablet_bootstrap.cc:654] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:08.813220 6534 tablet_bootstrap.cc:492] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Bootstrap starting.
I20260806 20:30:08.813512 6533 log.cc:824] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:08.815092 6531 tablet_bootstrap.cc:492] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: No bootstrap required, opened a new log
I20260806 20:30:08.815430 6531 ts_tablet_manager.cc:1397] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260806 20:30:08.816710 6534 tablet_bootstrap.cc:654] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:08.816847 6533 tablet_bootstrap.cc:492] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: No bootstrap required, opened a new log
I20260806 20:30:08.817118 6533 ts_tablet_manager.cc:1397] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260806 20:30:08.817775 6534 log.cc:824] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:08.820408 6534 tablet_bootstrap.cc:492] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: No bootstrap required, opened a new log
I20260806 20:30:08.820686 6534 ts_tablet_manager.cc:1397] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260806 20:30:08.826526 6531 raft_consensus.cc:348] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:08.827026 6531 raft_consensus.cc:374] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:08.827195 6531 raft_consensus.cc:729] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 951661b8953442c4846885cb17753c0e, State: Initialized, Role: FOLLOWER
I20260806 20:30:08.827908 6531 consensus_queue.cc:260] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:08.827841 6533 raft_consensus.cc:348] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:08.828279 6533 raft_consensus.cc:374] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:08.828394 6533 raft_consensus.cc:729] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a5fbc8eb4dd4b7880a4403cbdc54f4d, State: Initialized, Role: FOLLOWER
I20260806 20:30:08.828399 6534 raft_consensus.cc:348] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:08.828881 6534 raft_consensus.cc:374] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:08.829006 6534 raft_consensus.cc:729] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0cd971edad49a6a13f89a28077325d, State: Initialized, Role: FOLLOWER
I20260806 20:30:08.829160 6533 consensus_queue.cc:260] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:08.829600 6534 consensus_queue.cc:260] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:08.830549 6509 heartbeater.cc:503] Master 127.17.109.254:34529 was elected leader, sending a full tablet report...
I20260806 20:30:08.831244 6533 ts_tablet_manager.cc:1428] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260806 20:30:08.831719 6534 ts_tablet_manager.cc:1428] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260806 20:30:08.832299 6355 heartbeater.cc:503] Master 127.17.109.254:34529 was elected leader, sending a full tablet report...
I20260806 20:30:08.833434 6531 ts_tablet_manager.cc:1428] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Time spent starting tablet: real 0.018s user 0.008s sys 0.007s
W20260806 20:30:08.869510 6210 tablet.cc:2369] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:30:08.904385 6357 tablet.cc:2369] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:30:08.954294 6511 tablet.cc:2369] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:09.031463 6539 raft_consensus.cc:482] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:30:09.031862 6539 raft_consensus.cc:504] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:09.033625 6539 leader_election.cc:288] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc0cd971edad49a6a13f89a28077325d (127.17.109.193:38097), 5a5fbc8eb4dd4b7880a4403cbdc54f4d (127.17.109.195:43001)
I20260806 20:30:09.034652 6541 raft_consensus.cc:482] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:30:09.035109 6541 raft_consensus.cc:504] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:09.040704 6541 leader_election.cc:288] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 951661b8953442c4846885cb17753c0e (127.17.109.194:44963), 5a5fbc8eb4dd4b7880a4403cbdc54f4d (127.17.109.195:43001)
I20260806 20:30:09.043919 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.034098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:41377 (local address 127.17.109.193:38097)
0806 20:30:09.034303 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:09.034310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:09.035024 (+ 714us) server_negotiation.cc:407] Connection header received
0806 20:30:09.035424 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:09.035430 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:09.035516 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:09.035631 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:09.036286 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.037225 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.040541 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.042277 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.042450 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.042560 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:09.043344 (+ 784us) server_negotiation.cc:299] Negotiation successful
0806 20:30:09.043538 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:30:09.044143 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.034092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:35327 (local address 127.17.109.195:43001)
0806 20:30:09.034304 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:09.034310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:09.035410 (+ 1100us) server_negotiation.cc:407] Connection header received
0806 20:30:09.035660 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:09.035665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:09.035759 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:09.035840 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:09.036672 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.037773 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.041600 (+ 3827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.042923 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.043045 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.043117 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:09.043699 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 20:30:09.043867 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:30:09.044860 6543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.033928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:38097 (local address 127.17.109.194:41377)
0806 20:30:09.034897 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:09.034924 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:09.035164 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:09.035673 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:09.035680 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:09.035710 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:09.036116 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:09.036134 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.037407 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.037413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:09.040296 (+ 2883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:09.040317 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.042805 (+ 2488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.042814 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:09.043071 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.043123 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 20:30:09.044099 (+ 976us) client_negotiation.cc:240] Negotiation successful
0806 20:30:09.044278 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":2,"thread_start_us":1015,"threads_started":1}
I20260806 20:30:09.045935 6158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65908f5b0c6f420e9a9193ed5f457f4b" candidate_uuid: "951661b8953442c4846885cb17753c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0cd971edad49a6a13f89a28077325d" is_pre_election: true
I20260806 20:30:09.046311 6542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.034044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:43001 (local address 127.17.109.194:35327)
0806 20:30:09.035273 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:09.035296 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:09.035445 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:09.036075 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:09.036078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:09.036124 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:09.036489 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:09.036500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.037965 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.037977 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:09.041417 (+ 3440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:09.041429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.043307 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.043313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:09.043473 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.043518 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 20:30:09.045728 (+ 2210us) client_negotiation.cc:240] Negotiation successful
0806 20:30:09.045876 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1085,"mutex_wait_us":4,"thread_start_us":436,"threads_started":1}
I20260806 20:30:09.046446 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.038030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:60933 (local address 127.17.109.194:44963)
0806 20:30:09.038225 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:09.038235 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:09.038911 (+ 676us) server_negotiation.cc:407] Connection header received
0806 20:30:09.039191 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:09.039199 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:09.039275 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:09.039347 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:09.040150 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.041257 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.043853 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.045159 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.045289 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.045398 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:09.046022 (+ 624us) server_negotiation.cc:299] Negotiation successful
0806 20:30:09.046152 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 20:30:09.046437 6158 raft_consensus.cc:2454] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 951661b8953442c4846885cb17753c0e in term 0.
I20260806 20:30:09.047174 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.037617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:44963 (local address 127.17.109.193:60933)
0806 20:30:09.038805 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:09.038824 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:09.039031 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:09.039544 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:09.039548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:09.039593 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:09.039984 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:09.039997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.041450 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.041457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:09.043696 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:09.043711 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.045635 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.045643 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:09.045830 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.045864 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 20:30:09.046677 (+ 813us) client_negotiation.cc:240] Negotiation successful
0806 20:30:09.046824 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"thread_start_us":828,"threads_started":1}
I20260806 20:30:09.047187 6236 leader_election.cc:302] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 951661b8953442c4846885cb17753c0e, dc0cd971edad49a6a13f89a28077325d; no voters:
I20260806 20:30:09.047367 6459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65908f5b0c6f420e9a9193ed5f457f4b" candidate_uuid: "951661b8953442c4846885cb17753c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" is_pre_election: true
I20260806 20:30:09.047716 6539 raft_consensus.cc:2781] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:30:09.047951 6459 raft_consensus.cc:2454] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 951661b8953442c4846885cb17753c0e in term 0.
I20260806 20:30:09.048492 6307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65908f5b0c6f420e9a9193ed5f457f4b" candidate_uuid: "dc0cd971edad49a6a13f89a28077325d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "951661b8953442c4846885cb17753c0e" is_pre_election: true
I20260806 20:30:09.047870 6539 raft_consensus.cc:482] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:30:09.048938 6539 raft_consensus.cc:3037] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:09.051507 6546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.038182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:43001 (local address 127.17.109.193:44995)
0806 20:30:09.039146 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:09.039163 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:09.039250 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:09.044845 (+ 5595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:09.044850 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:09.044877 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:09.045175 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:09.045186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.047396 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.047401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:09.049368 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:09.049381 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.050966 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.050970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:09.051081 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.051105 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 20:30:09.051181 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 20:30:09.051282 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":632,"threads_started":1}
I20260806 20:30:09.051679 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.037722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:44995 (local address 127.17.109.195:43001)
0806 20:30:09.044376 (+ 6654us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:09.044382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:09.044398 (+ 16us) server_negotiation.cc:407] Connection header received
0806 20:30:09.044451 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:09.044456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:09.044529 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:09.044618 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:09.045378 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.047195 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.049524 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.050755 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.050916 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.050988 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:09.051240 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 20:30:09.051383 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6544,"thread_start_us":8849,"threads_started":1}
I20260806 20:30:09.052275 6539 raft_consensus.cc:504] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:09.052356 6459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65908f5b0c6f420e9a9193ed5f457f4b" candidate_uuid: "dc0cd971edad49a6a13f89a28077325d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" is_pre_election: true
I20260806 20:30:09.052568 6459 raft_consensus.cc:2454] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0cd971edad49a6a13f89a28077325d in term 0.
I20260806 20:30:09.052910 6307 raft_consensus.cc:2379] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc0cd971edad49a6a13f89a28077325d in current term 1: Already voted for candidate 951661b8953442c4846885cb17753c0e in this term.
I20260806 20:30:09.053061 6090 leader_election.cc:302] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a5fbc8eb4dd4b7880a4403cbdc54f4d, dc0cd971edad49a6a13f89a28077325d; no voters:
I20260806 20:30:09.053578 6541 raft_consensus.cc:2781] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:30:09.053629 6459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65908f5b0c6f420e9a9193ed5f457f4b" candidate_uuid: "951661b8953442c4846885cb17753c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d"
I20260806 20:30:09.053790 6541 raft_consensus.cc:482] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:30:09.053850 6459 raft_consensus.cc:3037] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:09.053946 6541 raft_consensus.cc:3037] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:09.054860 6158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65908f5b0c6f420e9a9193ed5f457f4b" candidate_uuid: "951661b8953442c4846885cb17753c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0cd971edad49a6a13f89a28077325d"
I20260806 20:30:09.056509 6459 raft_consensus.cc:2454] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 951661b8953442c4846885cb17753c0e in term 1.
I20260806 20:30:09.056703 6539 leader_election.cc:288] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [CANDIDATE]: Term 1 election: Requested vote from peers dc0cd971edad49a6a13f89a28077325d (127.17.109.193:38097), 5a5fbc8eb4dd4b7880a4403cbdc54f4d (127.17.109.195:43001)
I20260806 20:30:09.056914 6235 leader_election.cc:302] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a5fbc8eb4dd4b7880a4403cbdc54f4d, 951661b8953442c4846885cb17753c0e; no voters:
I20260806 20:30:09.057256 6539 raft_consensus.cc:2781] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:09.057268 6541 raft_consensus.cc:504] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:09.057778 6158 raft_consensus.cc:2379] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 951661b8953442c4846885cb17753c0e in current term 1: Already voted for candidate dc0cd971edad49a6a13f89a28077325d in this term.
I20260806 20:30:09.058291 6539 raft_consensus.cc:686] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 1 LEADER]: Becoming Leader. State: Replica: 951661b8953442c4846885cb17753c0e, State: Running, Role: LEADER
I20260806 20:30:09.058470 6541 leader_election.cc:288] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [CANDIDATE]: Term 1 election: Requested vote from peers 951661b8953442c4846885cb17753c0e (127.17.109.194:44963), 5a5fbc8eb4dd4b7880a4403cbdc54f4d (127.17.109.195:43001)
I20260806 20:30:09.059206 6539 consensus_queue.cc:237] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } }
I20260806 20:30:09.059478 6459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65908f5b0c6f420e9a9193ed5f457f4b" candidate_uuid: "dc0cd971edad49a6a13f89a28077325d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d"
I20260806 20:30:09.059803 6459 raft_consensus.cc:2379] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dc0cd971edad49a6a13f89a28077325d in current term 1: Already voted for candidate 951661b8953442c4846885cb17753c0e in this term.
I20260806 20:30:09.059803 6307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65908f5b0c6f420e9a9193ed5f457f4b" candidate_uuid: "dc0cd971edad49a6a13f89a28077325d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "951661b8953442c4846885cb17753c0e"
I20260806 20:30:09.060453 6089 leader_election.cc:302] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc0cd971edad49a6a13f89a28077325d; no voters: 5a5fbc8eb4dd4b7880a4403cbdc54f4d, 951661b8953442c4846885cb17753c0e
I20260806 20:30:09.062430 6541 raft_consensus.cc:2726] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260806 20:30:09.064656 6004 catalog_manager.cc:5495] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 951661b8953442c4846885cb17753c0e (127.17.109.194). New cstate: current_term: 1 leader_uuid: "951661b8953442c4846885cb17753c0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "951661b8953442c4846885cb17753c0e" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 44963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 } health_report { overall_health: UNKNOWN } } }
I20260806 20:30:09.082120 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.076256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46150 (local address 127.17.109.194:44963)
0806 20:30:09.076481 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:09.076487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:09.076618 (+ 131us) server_negotiation.cc:407] Connection header received
0806 20:30:09.076894 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:09.076898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:09.076989 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:09.077148 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:09.077560 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.078322 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.079884 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.080314 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.080459 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.080969 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:09.081482 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:09.081648 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:09.081718 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 20:30:09.081818 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260806 20:30:09.090696 6158 raft_consensus.cc:1261] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 1 FOLLOWER]: Refusing update from remote peer 951661b8953442c4846885cb17753c0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:09.090677 6459 raft_consensus.cc:1261] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 1 FOLLOWER]: Refusing update from remote peer 951661b8953442c4846885cb17753c0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:09.091635 6539 consensus_queue.cc:1035] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0cd971edad49a6a13f89a28077325d" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 38097 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:09.092048 6547 consensus_queue.cc:1035] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a5fbc8eb4dd4b7880a4403cbdc54f4d" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 43001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:09.104283 6550 mvcc.cc:205] Tried to move back new op lower bound from 7315653464424554496 to 7315653464312225792. Current Snapshot: MvccSnapshot[applied={T|T < 7315653464424554496}]
I20260806 20:30:09.120124 6004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:30:09.121384 6004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:30:09.125499 6004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27f4563ddcbd4bf0a1bb54e7adcdd908: Sending DeleteTablet for 3 replicas of tablet 65908f5b0c6f420e9a9193ed5f457f4b
I20260806 20:30:09.126704 6285 tablet_service.cc:1514] Processing DeleteTablet for tablet 65908f5b0c6f420e9a9193ed5f457f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:09 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:46136
I20260806 20:30:09.126691 6433 tablet_service.cc:1514] Processing DeleteTablet for tablet 65908f5b0c6f420e9a9193ed5f457f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:09 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:57996
I20260806 20:30:09.126816 6138 tablet_service.cc:1514] Processing DeleteTablet for tablet 65908f5b0c6f420e9a9193ed5f457f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:09 UTC) from {username='kudu', principal='kudu/127.17.109.254@KRBTEST.COM'} at 127.0.0.1:50368
I20260806 20:30:09.133562 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 6068
I20260806 20:30:09.177310 6563 tablet_replica.cc:330] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: stopping tablet replica
I20260806 20:30:09.177781 6563 raft_consensus.cc:2229] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:30:09.178303 6563 raft_consensus.cc:2258] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:09.179190 6562 tablet_replica.cc:330] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: stopping tablet replica
I20260806 20:30:09.179613 6562 raft_consensus.cc:2229] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:30:09.180241 6562 raft_consensus.cc:2258] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:09.185688 6562 ts_tablet_manager.cc:1905] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:09.186419 6564 tablet_replica.cc:330] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: stopping tablet replica
I20260806 20:30:09.186856 6564 raft_consensus.cc:2229] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:30:09.187388 6564 raft_consensus.cc:2258] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:09.188900 6563 ts_tablet_manager.cc:1905] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:09.195966 6563 ts_tablet_manager.cc:1918] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:09.196141 6563 log.cc:1196] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/65908f5b0c6f420e9a9193ed5f457f4b
I20260806 20:30:09.196781 6563 ts_tablet_manager.cc:1939] T 65908f5b0c6f420e9a9193ed5f457f4b P 5a5fbc8eb4dd4b7880a4403cbdc54f4d: Deleting consensus metadata
I20260806 20:30:09.197378 6564 ts_tablet_manager.cc:1905] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:09.198930 5989 catalog_manager.cc:4841] TS 5a5fbc8eb4dd4b7880a4403cbdc54f4d (127.17.109.195:43001): tablet 65908f5b0c6f420e9a9193ed5f457f4b (table test-table [id=a8b3a968258d48659df624e25f8c455a]) successfully deleted
Aug 06 20:30:09 dist-test-slave-6k22 krb5kdc[5956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.193@KRBTEST.COM
I20260806 20:30:09.200482 6562 ts_tablet_manager.cc:1918] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:09.200776 6562 log.cc:1196] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/65908f5b0c6f420e9a9193ed5f457f4b
I20260806 20:30:09.201493 6562 ts_tablet_manager.cc:1939] T 65908f5b0c6f420e9a9193ed5f457f4b P 951661b8953442c4846885cb17753c0e: Deleting consensus metadata
I20260806 20:30:09.205430 6564 ts_tablet_manager.cc:1918] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:09.205657 5988 catalog_manager.cc:4841] TS 951661b8953442c4846885cb17753c0e (127.17.109.194:44963): tablet 65908f5b0c6f420e9a9193ed5f457f4b (table test-table [id=a8b3a968258d48659df624e25f8c455a]) successfully deleted
I20260806 20:30:09.205701 6564 log.cc:1196] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786047878953513-17847-0/minicluster-data/ts-0/wal/wals/65908f5b0c6f420e9a9193ed5f457f4b
I20260806 20:30:09.206346 6564 ts_tablet_manager.cc:1939] T 65908f5b0c6f420e9a9193ed5f457f4b P dc0cd971edad49a6a13f89a28077325d: Deleting consensus metadata
I20260806 20:30:09.208307 5990 catalog_manager.cc:4841] TS dc0cd971edad49a6a13f89a28077325d (127.17.109.193:38097): tablet 65908f5b0c6f420e9a9193ed5f457f4b (table test-table [id=a8b3a968258d48659df624e25f8c455a]) successfully deleted
I20260806 20:30:09.209760 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.135218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.17.109.193:38097)
0806 20:30:09.135454 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:09.135463 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:09.166201 (+ 30738us) server_negotiation.cc:407] Connection header received
0806 20:30:09.166844 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:09.166853 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:09.166971 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:09.167065 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:09.171785 (+ 4720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.173548 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.183797 (+ 10249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.184509 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.184646 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.200822 (+ 16176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:09.200838 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:09.200857 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:09.200913 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:09.205169 (+ 4256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:09.206943 (+ 1774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:09.206952 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:09.206956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:09.207156 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:09.207765 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:09.207770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:09.207774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:09.208243 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:09.208529 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:09.209119 (+ 590us) server_negotiation.cc:299] Negotiation successful
0806 20:30:09.209378 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:30:09.211365 6204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:09.303211 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 6068
I20260806 20:30:09.327286 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 6215
Aug 06 20:30:09 dist-test-slave-6k22 krb5kdc[5956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.194@KRBTEST.COM
I20260806 20:30:09.346621 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.328159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46160 (local address 127.17.109.194:44963)
0806 20:30:09.328406 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:09.328412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:09.328473 (+ 61us) server_negotiation.cc:407] Connection header received
0806 20:30:09.328661 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:09.328667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:09.328758 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:09.328876 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:09.330371 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.331493 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.333153 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.333814 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.333946 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.339044 (+ 5098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:09.339060 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:09.339082 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:09.339145 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:09.343273 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:09.344122 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:09.344132 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:09.344136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:09.344341 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:09.344766 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:09.344772 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:09.344778 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:09.345297 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:09.345616 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:09.345903 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 20:30:09.346140 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260806 20:30:09.347337 6350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:09.467410 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 6215
I20260806 20:30:09.488530 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 6362
Aug 06 20:30:09 dist-test-slave-6k22 krb5kdc[5956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.195@KRBTEST.COM
I20260806 20:30:09.506155 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:09.489497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.17.109.195:43001)
0806 20:30:09.489707 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:09.489712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:09.489732 (+ 20us) server_negotiation.cc:407] Connection header received
0806 20:30:09.489914 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:09.489918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:09.490000 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:09.490093 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:09.491832 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.492601 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.494203 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:09.494807 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:09.494948 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:09.500349 (+ 5401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:09.500367 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:09.500384 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:09.500426 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:09.503335 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:09.504206 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:09.504214 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:09.504216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:09.504346 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:09.504710 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:09.504713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:09.504715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:09.505087 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:09.505300 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:09.505631 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 20:30:09.505864 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 20:30:09.506883 6503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:09.600333 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 6362
I20260806 20:30:09.622418 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 5971
I20260806 20:30:09.623725 6035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:09.741796 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 5971
2026-08-06T20:30:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4808 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T20:30:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:09Z Disabled control of system clock
I20260806 20:30:09.836535 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:43741
--webserver_interface=127.17.109.254
--webserver_port=0
--builtin_ntp_servers=127.17.109.212:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:43741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 20:30:10.189438 6574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:10.189745 6574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:10.189831 6574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:10.197263 6574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:10.197372 6574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:10.197412 6574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:10.197471 6574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:10.197516 6574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:10.207095 6574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:43741
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:43741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:10.208459 6574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:10.210254 6574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:10.222556 6579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:10.222576 6582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:10.222519 6580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:10.223580 6574 server_base.cc:1034] running on GCE node
I20260806 20:30:10.224434 6574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:10.225881 6574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:10.227141 6574 hybrid_clock.cc:648] HybridClock initialized: now 1786048210227108 us; error 55 us; skew 500 ppm
I20260806 20:30:10.227527 6574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:30:10.229969 6574 webserver.cc:460] Webserver started at http://127.17.109.254:38267/ using document root <none> and password file <none>
I20260806 20:30:10.230675 6574 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:10.230823 6574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:10.231150 6574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:10.234150 6574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "22b3945a1e0c4158b30d6e7088c3f300"
format_stamp: "Formatted at 2026-08-06 20:30:10 on dist-test-slave-6k22"
server_key: "3b48052e0e7155ffbae95186ace1e40e"
server_key_iv: "d3535f3222d0feefed7958b59b225367"
server_key_version: "encryptionkey@0"
I20260806 20:30:10.234984 6574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "22b3945a1e0c4158b30d6e7088c3f300"
format_stamp: "Formatted at 2026-08-06 20:30:10 on dist-test-slave-6k22"
server_key: "3b48052e0e7155ffbae95186ace1e40e"
server_key_iv: "d3535f3222d0feefed7958b59b225367"
server_key_version: "encryptionkey@0"
I20260806 20:30:10.240302 6574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 20:30:10.243960 6589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:10.245431 6574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:30:10.245675 6574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "22b3945a1e0c4158b30d6e7088c3f300"
format_stamp: "Formatted at 2026-08-06 20:30:10 on dist-test-slave-6k22"
server_key: "3b48052e0e7155ffbae95186ace1e40e"
server_key_iv: "d3535f3222d0feefed7958b59b225367"
server_key_version: "encryptionkey@0"
I20260806 20:30:10.245944 6574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:10.264779 6574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:10.265806 6574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:10.266180 6574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:10.287734 6574 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:43741
I20260806 20:30:10.287746 6645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:43741 every 8 connection(s)
I20260806 20:30:10.289414 6574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:10.293339 6646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:10.297992 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 6574
I20260806 20:30:10.298177 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:10.298629 17847 external_mini_cluster.cc:1428] Setting key 11622f04245b7fd590c37bac86cbce24
I20260806 20:30:10.304440 6646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300: Bootstrap starting.
I20260806 20:30:10.308105 6646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:10.309303 6646 log.cc:824] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:10.312577 6646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300: No bootstrap required, opened a new log
I20260806 20:30:10.313030 6649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:10.300640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38864 (local address 127.17.109.254:43741)
0806 20:30:10.301738 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:10.301749 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:10.301794 (+ 45us) server_negotiation.cc:407] Connection header received
0806 20:30:10.303537 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:10.303625 (+ 88us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:10.304088 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:10.304411 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:10.305080 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:10.306431 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:10.308387 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:10.309061 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:10.309192 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:10.310544 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:10.310554 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:10.310573 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:10.311174 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:10.311261 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:10.311282 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:10.311369 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:10.311671 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 20:30:10.312016 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":411,"threads_started":1}
I20260806 20:30:10.320669 6646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b3945a1e0c4158b30d6e7088c3f300" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 43741 } }
I20260806 20:30:10.321029 6646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:10.321157 6646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22b3945a1e0c4158b30d6e7088c3f300, State: Initialized, Role: FOLLOWER
I20260806 20:30:10.321694 6646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b3945a1e0c4158b30d6e7088c3f300" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 43741 } }
I20260806 20:30:10.321947 6646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:10.322110 6646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:10.322296 6646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:10.324577 6646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b3945a1e0c4158b30d6e7088c3f300" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 43741 } }
I20260806 20:30:10.325117 6646 leader_election.cc:302] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22b3945a1e0c4158b30d6e7088c3f300; no voters:
I20260806 20:30:10.325807 6646 leader_election.cc:288] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:10.325978 6651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:10.326737 6651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [term 1 LEADER]: Becoming Leader. State: Replica: 22b3945a1e0c4158b30d6e7088c3f300, State: Running, Role: LEADER
I20260806 20:30:10.327230 6651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b3945a1e0c4158b30d6e7088c3f300" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 43741 } }
I20260806 20:30:10.328064 6646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:10.330852 6652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22b3945a1e0c4158b30d6e7088c3f300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b3945a1e0c4158b30d6e7088c3f300" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 43741 } } }
I20260806 20:30:10.331321 6652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:10.331410 6653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22b3945a1e0c4158b30d6e7088c3f300. Latest consensus state: current_term: 1 leader_uuid: "22b3945a1e0c4158b30d6e7088c3f300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b3945a1e0c4158b30d6e7088c3f300" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 43741 } } }
I20260806 20:30:10.331840 6653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:10.336236 6658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:10.342345 6658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:10.352137 6658 catalog_manager.cc:1347] Generated new cluster ID: 723922c5710245a7bb0613724f510e38
I20260806 20:30:10.352272 6658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:10.381103 6658 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:10.383049 6658 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:10.400159 6658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300: Generated new TSK 0
I20260806 20:30:10.401377 6658 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:10.413277 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.193:0
--local_ip_for_outbound_sockets=127.17.109.193
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:43741
--builtin_ntp_servers=127.17.109.212:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 20:30:10.768110 6673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:10.768389 6673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:10.768476 6673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:10.776047 6673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:10.776158 6673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:10.776266 6673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.193
I20260806 20:30:10.785485 6673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.109.193
--webserver_port=0
--tserver_master_addrs=127.17.109.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.193
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:10.786777 6673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:10.788616 6673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:10.806655 6679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:10.806754 6682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:10.806684 6680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:10.807698 6673 server_base.cc:1034] running on GCE node
I20260806 20:30:10.808477 6673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:10.809571 6673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:10.810864 6673 hybrid_clock.cc:648] HybridClock initialized: now 1786048210810820 us; error 64 us; skew 500 ppm
I20260806 20:30:10.811283 6673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:30:10.813982 6673 webserver.cc:460] Webserver started at http://127.17.109.193:42861/ using document root <none> and password file <none>
I20260806 20:30:10.814710 6673 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:10.814854 6673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:10.815191 6673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:10.818210 6673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/data/instance:
uuid: "905e8fb9e38d4866993805f1fa379455"
format_stamp: "Formatted at 2026-08-06 20:30:10 on dist-test-slave-6k22"
server_key: "4f4b4799fdda388ccd025f23b015cd59"
server_key_iv: "b2ea6a8fdd4c0b0e91a8071dfdb21537"
server_key_version: "encryptionkey@0"
I20260806 20:30:10.819069 6673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance:
uuid: "905e8fb9e38d4866993805f1fa379455"
format_stamp: "Formatted at 2026-08-06 20:30:10 on dist-test-slave-6k22"
server_key: "4f4b4799fdda388ccd025f23b015cd59"
server_key_iv: "b2ea6a8fdd4c0b0e91a8071dfdb21537"
server_key_version: "encryptionkey@0"
I20260806 20:30:10.824343 6673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 20:30:10.827999 6688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:10.829510 6673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:30:10.829682 6673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/wal
uuid: "905e8fb9e38d4866993805f1fa379455"
format_stamp: "Formatted at 2026-08-06 20:30:10 on dist-test-slave-6k22"
server_key: "4f4b4799fdda388ccd025f23b015cd59"
server_key_iv: "b2ea6a8fdd4c0b0e91a8071dfdb21537"
server_key_version: "encryptionkey@0"
I20260806 20:30:10.829917 6673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:10.854650 6673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:10.855541 6673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:10.855943 6673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:10.856954 6673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:10.858497 6673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:10.858625 6673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:10.858757 6673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:10.858856 6673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:10.892673 6673 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.193:37273
I20260806 20:30:10.892776 6812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.193:37273 every 8 connection(s)
I20260806 20:30:10.894057 6673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/data/info.pb
I20260806 20:30:10.897372 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 6673
I20260806 20:30:10.897547 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-0/wal/instance
I20260806 20:30:10.898169 17847 external_mini_cluster.cc:1428] Setting key 65616db3d7f012a6e72875099a3fe773
I20260806 20:30:10.901778 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.194:0
--local_ip_for_outbound_sockets=127.17.109.194
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:43741
--builtin_ntp_servers=127.17.109.212:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 20:30:10.909258 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:10.896771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:60951 (local address 127.17.109.254:43741)
0806 20:30:10.897302 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:10.897311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:10.898803 (+ 1492us) server_negotiation.cc:407] Connection header received
0806 20:30:10.901089 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:10.901093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:10.901156 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:10.901251 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:10.902775 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:10.903712 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:10.906186 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:10.906595 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:10.906692 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:10.908415 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:10.908420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:10.908424 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:10.908444 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:10.908487 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:10.908501 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:10.908589 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:10.908870 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 20:30:10.908955 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":267,"threads_started":1}
I20260806 20:30:10.910687 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:10.897360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:43741 (local address 127.17.109.193:60951)
0806 20:30:10.898575 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:10.898632 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:10.900781 (+ 2149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:10.901526 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:10.901538 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:10.901700 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:10.902557 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:10.902586 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:10.903916 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:10.903925 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:10.906030 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:10.906047 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:10.906836 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:10.906846 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:10.907183 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:10.908009 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:30:10.908014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:10.908068 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:30:10.908179 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:10.908617 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:10.908630 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:10.908640 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 20:30:10.908833 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 20:30:10.909262 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":365,"threads_started":1}
I20260806 20:30:10.912501 6813 heartbeater.cc:347] Connected to a master server at 127.17.109.254:43741
I20260806 20:30:10.912843 6813 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:10.913753 6813 heartbeater.cc:511] Master 127.17.109.254:43741 requested a full tablet report, sending...
I20260806 20:30:10.916066 6607 ts_manager.cc:194] Registered new tserver with Master: 905e8fb9e38d4866993805f1fa379455 (127.17.109.193:37273)
I20260806 20:30:10.917516 6607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.193:60951
W20260806 20:30:11.242314 6817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:11.242663 6817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:11.242841 6817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:11.250187 6817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:11.250361 6817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:11.250559 6817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.194
I20260806 20:30:11.259742 6817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.109.194
--webserver_port=0
--tserver_master_addrs=127.17.109.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.194
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:11.261133 6817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:11.262945 6817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:11.276523 6825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:11.276594 6823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:11.276561 6822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:11.277349 6817 server_base.cc:1034] running on GCE node
I20260806 20:30:11.277997 6817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:11.278923 6817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:11.280131 6817 hybrid_clock.cc:648] HybridClock initialized: now 1786048211280097 us; error 29 us; skew 500 ppm
I20260806 20:30:11.280457 6817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:30:11.282856 6817 webserver.cc:460] Webserver started at http://127.17.109.194:43541/ using document root <none> and password file <none>
I20260806 20:30:11.283627 6817 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:11.283756 6817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:11.284096 6817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:11.287071 6817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/data/instance:
uuid: "6464752f0e0e4c7bbd6e9063bccb9609"
format_stamp: "Formatted at 2026-08-06 20:30:11 on dist-test-slave-6k22"
server_key: "4e3349eab51ffe8cb1fa61984ca2b2de"
server_key_iv: "4230e0544ff9f7756f2e86ab9e554586"
server_key_version: "encryptionkey@0"
I20260806 20:30:11.287974 6817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance:
uuid: "6464752f0e0e4c7bbd6e9063bccb9609"
format_stamp: "Formatted at 2026-08-06 20:30:11 on dist-test-slave-6k22"
server_key: "4e3349eab51ffe8cb1fa61984ca2b2de"
server_key_iv: "4230e0544ff9f7756f2e86ab9e554586"
server_key_version: "encryptionkey@0"
I20260806 20:30:11.293386 6817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:30:11.297092 6831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:11.298532 6817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:11.298699 6817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/wal
uuid: "6464752f0e0e4c7bbd6e9063bccb9609"
format_stamp: "Formatted at 2026-08-06 20:30:11 on dist-test-slave-6k22"
server_key: "4e3349eab51ffe8cb1fa61984ca2b2de"
server_key_iv: "4230e0544ff9f7756f2e86ab9e554586"
server_key_version: "encryptionkey@0"
I20260806 20:30:11.298897 6817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:11.317283 6817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:11.318111 6817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:11.318462 6817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:11.319447 6817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:11.320906 6817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:11.320996 6817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:11.321156 6817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:11.321249 6817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:11.356397 6817 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.194:45609
I20260806 20:30:11.356523 6960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.194:45609 every 8 connection(s)
I20260806 20:30:11.357780 6817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/data/info.pb
I20260806 20:30:11.365887 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 6817
I20260806 20:30:11.366081 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/wal/instance
I20260806 20:30:11.366582 17847 external_mini_cluster.cc:1428] Setting key 641963c09f35d4a69bd04bb2668898f4
I20260806 20:30:11.369936 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.109.195:0
--local_ip_for_outbound_sockets=127.17.109.195
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:43741
--builtin_ntp_servers=127.17.109.212:42945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 20:30:11.371150 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.360609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.194:41701 (local address 127.17.109.254:43741)
0806 20:30:11.360855 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:11.360861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:11.362217 (+ 1356us) server_negotiation.cc:407] Connection header received
0806 20:30:11.364003 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:11.364007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:11.364076 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:11.364166 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:11.365170 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.366219 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.368024 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.368671 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.368819 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.370378 (+ 1559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:11.370383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:11.370386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:11.370403 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:11.370434 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:11.370445 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:11.370515 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:11.370833 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 20:30:11.370918 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 20:30:11.372684 6963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.361189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:43741 (local address 127.17.109.194:41701)
0806 20:30:11.362063 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:11.362100 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:11.363758 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:11.364383 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:11.364391 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:11.364474 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:11.365025 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.365040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.366364 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.366368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.367883 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.367895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.368929 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.368938 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.369230 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.369998 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:30:11.370003 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:11.370060 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:30:11.370177 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:11.370551 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:11.370564 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:11.370576 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 20:30:11.370766 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 20:30:11.371216 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":266,"threads_started":1}
I20260806 20:30:11.374289 6961 heartbeater.cc:347] Connected to a master server at 127.17.109.254:43741
I20260806 20:30:11.374624 6961 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:11.375468 6961 heartbeater.cc:511] Master 127.17.109.254:43741 requested a full tablet report, sending...
I20260806 20:30:11.376991 6607 ts_manager.cc:194] Registered new tserver with Master: 6464752f0e0e4c7bbd6e9063bccb9609 (127.17.109.194:45609)
I20260806 20:30:11.377909 6607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.194:41701
W20260806 20:30:11.715003 6964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:11.715350 6964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:11.715513 6964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:11.723043 6964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:11.723193 6964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:11.723380 6964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.109.195
I20260806 20:30:11.732882 6964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:42945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.109.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.109.195
--webserver_port=0
--tserver_master_addrs=127.17.109.254:43741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.109.195
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:11.734377 6964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:11.736400 6964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:11.750118 6970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:11.750144 6971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:11.750943 6973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:11.751073 6964 server_base.cc:1034] running on GCE node
I20260806 20:30:11.751809 6964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:11.752914 6964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:11.754168 6964 hybrid_clock.cc:648] HybridClock initialized: now 1786048211754121 us; error 43 us; skew 500 ppm
I20260806 20:30:11.754518 6964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 20:30:11.757084 6964 webserver.cc:460] Webserver started at http://127.17.109.195:33463/ using document root <none> and password file <none>
I20260806 20:30:11.757865 6964 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:11.758015 6964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:11.758407 6964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:11.761735 6964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/data/instance:
uuid: "8016649893fd4aaabf03e1a7aba14d19"
format_stamp: "Formatted at 2026-08-06 20:30:11 on dist-test-slave-6k22"
server_key: "b3d135d2d8f4ae195b8aca70cb4c80c9"
server_key_iv: "40a598fcd5ef5b957948798b852527d5"
server_key_version: "encryptionkey@0"
I20260806 20:30:11.762627 6964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance:
uuid: "8016649893fd4aaabf03e1a7aba14d19"
format_stamp: "Formatted at 2026-08-06 20:30:11 on dist-test-slave-6k22"
server_key: "b3d135d2d8f4ae195b8aca70cb4c80c9"
server_key_iv: "40a598fcd5ef5b957948798b852527d5"
server_key_version: "encryptionkey@0"
I20260806 20:30:11.768260 6964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 20:30:11.772004 6979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:11.773346 6964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:11.773533 6964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/wal
uuid: "8016649893fd4aaabf03e1a7aba14d19"
format_stamp: "Formatted at 2026-08-06 20:30:11 on dist-test-slave-6k22"
server_key: "b3d135d2d8f4ae195b8aca70cb4c80c9"
server_key_iv: "40a598fcd5ef5b957948798b852527d5"
server_key_version: "encryptionkey@0"
I20260806 20:30:11.773772 6964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:11.800712 6964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:11.801570 6964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:11.801913 6964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:11.803094 6964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 20:30:11.804817 6964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 20:30:11.804947 6964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:11.805086 6964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 20:30:11.805190 6964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:11.839252 6964 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.195:40889
I20260806 20:30:11.839349 7109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.195:40889 every 8 connection(s)
I20260806 20:30:11.840615 6964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/data/info.pb
I20260806 20:30:11.845098 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 6964
I20260806 20:30:11.845285 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/wal/instance
I20260806 20:30:11.845870 17847 external_mini_cluster.cc:1428] Setting key 99fb1ff8f2de843371a0e05ae166aae3
I20260806 20:30:11.853737 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.843282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.195:46523 (local address 127.17.109.254:43741)
0806 20:30:11.843469 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:11.843474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:11.845137 (+ 1663us) server_negotiation.cc:407] Connection header received
0806 20:30:11.847273 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:11.847277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:11.847344 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:11.847421 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:11.848548 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.849315 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.850947 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.851348 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.851493 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.852840 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:11.852845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:11.852848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:11.852864 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:11.852895 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:11.852906 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:11.853011 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:11.853246 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 20:30:11.853346 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 20:30:11.854651 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.843913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:43741 (local address 127.17.109.195:46523)
0806 20:30:11.844942 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:11.845006 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:11.846998 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:11.847727 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:11.847735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:11.847826 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:11.848372 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.848388 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.849474 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.849480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.850779 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.850790 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.851604 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.851612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.851839 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.852486 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 20:30:11.852489 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:11.852526 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 20:30:11.852602 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:11.853010 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:11.853022 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:11.853033 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 20:30:11.853220 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 20:30:11.853606 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":290,"threads_started":1}
I20260806 20:30:11.856120 7110 heartbeater.cc:347] Connected to a master server at 127.17.109.254:43741
I20260806 20:30:11.856451 7110 heartbeater.cc:464] Registering TS with master...
I20260806 20:30:11.857208 7110 heartbeater.cc:511] Master 127.17.109.254:43741 requested a full tablet report, sending...
I20260806 20:30:11.858682 6607 ts_manager.cc:194] Registered new tserver with Master: 8016649893fd4aaabf03e1a7aba14d19 (127.17.109.195:40889)
I20260806 20:30:11.859791 6607 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.109.195:46523
I20260806 20:30:11.861491 17847 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 20:30:11.871671 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.865544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38878 (local address 127.17.109.254:43741)
0806 20:30:11.865715 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:11.865719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:11.866226 (+ 507us) server_negotiation.cc:407] Connection header received
0806 20:30:11.866364 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:11.866368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:11.866440 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:11.866525 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:11.866999 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.867737 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.869104 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.869521 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.869612 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.870876 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:11.870880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:11.870883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:11.870901 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:11.870930 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:11.870941 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:11.871012 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:11.871186 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 20:30:11.871297 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 20:30:11.875931 6607 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 20:30:11.883023 6607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 20:30:11.915874 7123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.896371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45609 (local address 127.0.0.1:56880)
0806 20:30:11.898977 (+ 2606us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:11.899016 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:11.899193 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:11.908365 (+ 9172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:11.908376 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:11.908412 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:11.909051 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.909071 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.911010 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.911016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.913000 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.913013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.914850 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.914856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.915040 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.915090 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 20:30:11.915251 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 20:30:11.915446 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2372,"mutex_wait_us":76,"thread_start_us":9919,"threads_started":1}
I20260806 20:30:11.915853 7124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.896933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56880 (local address 127.17.109.194:45609)
0806 20:30:11.907494 (+ 10561us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:11.907504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:11.907541 (+ 37us) server_negotiation.cc:407] Connection header received
0806 20:30:11.907676 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:11.907687 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:11.907983 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:11.908235 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:11.909257 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.910853 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.913213 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.914647 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.914839 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.915064 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:11.915299 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 20:30:11.915465 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10178,"thread_start_us":10059,"threads_started":1}
I20260806 20:30:11.917179 7122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.896489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37273 (local address 127.0.0.1:34178)
0806 20:30:11.906452 (+ 9963us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:11.906478 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:11.906627 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:11.909092 (+ 2465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:11.909103 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:11.909136 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:11.909500 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.909508 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.911397 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.911403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.913789 (+ 2386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.913812 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.916232 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.916239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.916455 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.916492 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 20:30:11.916603 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 20:30:11.916745 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9766,"mutex_wait_us":4,"thread_start_us":2079,"threads_started":1}
I20260806 20:30:11.917198 7125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.897156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.17.109.193:37273)
0806 20:30:11.908490 (+ 11334us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:11.908500 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:11.908530 (+ 30us) server_negotiation.cc:407] Connection header received
0806 20:30:11.908636 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:11.908645 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:11.908870 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:11.909084 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:11.909711 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.911233 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.914064 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.915622 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.915840 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.916067 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:11.916664 (+ 597us) server_negotiation.cc:299] Negotiation successful
0806 20:30:11.916839 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11036,"thread_start_us":10899,"threads_started":1}
I20260806 20:30:11.917711 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.899388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:40889 (local address 127.0.0.1:43752)
0806 20:30:11.907678 (+ 8290us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:11.907698 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:11.907834 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:11.908558 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:11.908565 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:11.908599 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:11.909979 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.909997 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.912046 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.912057 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.914371 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:11.914387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.916772 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.916782 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:11.916963 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.917001 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 20:30:11.917110 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 20:30:11.917263 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8140,"mutex_wait_us":784,"thread_start_us":7985,"threads_started":1}
I20260806 20:30:11.918648 7127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:11.899820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43752 (local address 127.17.109.195:40889)
0806 20:30:11.906237 (+ 6417us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:11.906252 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:11.907838 (+ 1586us) server_negotiation.cc:407] Connection header received
0806 20:30:11.907997 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:11.908009 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:11.908239 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:11.908442 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:11.910186 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.911898 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.914612 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:11.916108 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:11.916327 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:11.916530 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:11.917967 (+ 1437us) server_negotiation.cc:299] Negotiation successful
0806 20:30:11.918143 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6140,"thread_start_us":5915,"threads_started":1}
I20260806 20:30:11.920697 6813 heartbeater.cc:503] Master 127.17.109.254:43741 was elected leader, sending a full tablet report...
I20260806 20:30:11.920212 6884 tablet_service.cc:1467] Processing CreateTablet for tablet 276d54b1855049ecbf1687e0eee1281b (DEFAULT_TABLE table=test-table [id=a1befc2ff39c4de49f43ae4be0273094]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:11.921730 6884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 276d54b1855049ecbf1687e0eee1281b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:11.922485 7033 tablet_service.cc:1467] Processing CreateTablet for tablet 276d54b1855049ecbf1687e0eee1281b (DEFAULT_TABLE table=test-table [id=a1befc2ff39c4de49f43ae4be0273094]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:11.922489 6737 tablet_service.cc:1467] Processing CreateTablet for tablet 276d54b1855049ecbf1687e0eee1281b (DEFAULT_TABLE table=test-table [id=a1befc2ff39c4de49f43ae4be0273094]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 20:30:11.923981 6737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 276d54b1855049ecbf1687e0eee1281b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:11.924194 7033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 276d54b1855049ecbf1687e0eee1281b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:11.936007 7128 tablet_bootstrap.cc:492] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Bootstrap starting.
I20260806 20:30:11.939007 7130 tablet_bootstrap.cc:492] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: Bootstrap starting.
I20260806 20:30:11.941291 7128 tablet_bootstrap.cc:654] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:11.942430 7129 tablet_bootstrap.cc:492] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Bootstrap starting.
I20260806 20:30:11.942701 7128 log.cc:824] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:11.942934 7130 tablet_bootstrap.cc:654] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:11.944576 7130 log.cc:824] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:11.945647 7129 tablet_bootstrap.cc:654] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:11.949182 7129 log.cc:824] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:11.950479 7130 tablet_bootstrap.cc:492] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: No bootstrap required, opened a new log
I20260806 20:30:11.950659 7128 tablet_bootstrap.cc:492] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: No bootstrap required, opened a new log
I20260806 20:30:11.950843 7130 ts_tablet_manager.cc:1397] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260806 20:30:11.950939 7128 ts_tablet_manager.cc:1397] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260806 20:30:11.952477 7129 tablet_bootstrap.cc:492] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: No bootstrap required, opened a new log
I20260806 20:30:11.952744 7129 ts_tablet_manager.cc:1397] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260806 20:30:11.959471 7130 raft_consensus.cc:348] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:11.959978 7130 raft_consensus.cc:374] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:11.960140 7130 raft_consensus.cc:729] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 905e8fb9e38d4866993805f1fa379455, State: Initialized, Role: FOLLOWER
I20260806 20:30:11.960853 7130 consensus_queue.cc:260] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:11.961755 7128 raft_consensus.cc:348] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:11.962196 7128 raft_consensus.cc:374] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:11.962312 7128 raft_consensus.cc:729] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8016649893fd4aaabf03e1a7aba14d19, State: Initialized, Role: FOLLOWER
I20260806 20:30:11.962826 7130 ts_tablet_manager.cc:1428] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260806 20:30:11.963002 7128 consensus_queue.cc:260] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:11.963107 7129 raft_consensus.cc:348] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:11.963783 7129 raft_consensus.cc:374] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:11.963938 7129 raft_consensus.cc:729] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6464752f0e0e4c7bbd6e9063bccb9609, State: Initialized, Role: FOLLOWER
I20260806 20:30:11.964298 7110 heartbeater.cc:503] Master 127.17.109.254:43741 was elected leader, sending a full tablet report...
I20260806 20:30:11.965009 7128 ts_tablet_manager.cc:1428] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260806 20:30:11.965826 7129 consensus_queue.cc:260] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:11.967757 6961 heartbeater.cc:503] Master 127.17.109.254:43741 was elected leader, sending a full tablet report...
I20260806 20:30:11.968569 7129 ts_tablet_manager.cc:1428] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260806 20:30:12.072765 7135 raft_consensus.cc:482] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 20:30:12.073169 7135 raft_consensus.cc:504] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:12.075345 7135 leader_election.cc:288] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6464752f0e0e4c7bbd6e9063bccb9609 (127.17.109.194:45609), 8016649893fd4aaabf03e1a7aba14d19 (127.17.109.195:40889)
I20260806 20:30:12.084089 7124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.075829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:47923 (local address 127.17.109.194:45609)
0806 20:30:12.076368 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:12.076375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:12.076563 (+ 188us) server_negotiation.cc:407] Connection header received
0806 20:30:12.076832 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:12.076838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:12.076948 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:12.077079 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:12.077742 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.078671 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.081883 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.083047 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.083206 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:12.083290 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:12.083716 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 20:30:12.083840 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395}
I20260806 20:30:12.084314 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.075594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.194:45609 (local address 127.17.109.193:47923)
0806 20:30:12.076403 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.076429 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:12.076631 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:12.077147 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:12.077154 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:12.077185 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:12.077603 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:12.077617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.078830 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.078835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:12.081657 (+ 2822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:12.081675 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.083272 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.083278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:12.083418 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:12.083494 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 20:30:12.083664 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 20:30:12.083823 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":115,"thread_start_us":446,"threads_started":1}
I20260806 20:30:12.084553 7127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.076091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.109.193:52713 (local address 127.17.109.195:40889)
0806 20:30:12.076333 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:12.076344 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:12.077119 (+ 775us) server_negotiation.cc:407] Connection header received
0806 20:30:12.077337 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:12.077345 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:12.077424 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:12.077529 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 20:30:12.078308 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.079194 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.082132 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.083380 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.083519 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:12.083628 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:12.084090 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 20:30:12.084247 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 20:30:12.084760 7140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.075901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.195:40889 (local address 127.17.109.193:52713)
0806 20:30:12.076989 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.077011 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:12.077169 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:12.077578 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:12.077583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:12.077608 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 20:30:12.078089 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:12.078100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.079360 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.079373 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:12.081926 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:12.081941 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.083669 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.083679 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:12.083889 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:12.083927 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 20:30:12.084037 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 20:30:12.084181 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"thread_start_us":516,"threads_started":1}
I20260806 20:30:12.085574 6910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "276d54b1855049ecbf1687e0eee1281b" candidate_uuid: "905e8fb9e38d4866993805f1fa379455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" is_pre_election: true
I20260806 20:30:12.085964 7057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "276d54b1855049ecbf1687e0eee1281b" candidate_uuid: "905e8fb9e38d4866993805f1fa379455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8016649893fd4aaabf03e1a7aba14d19" is_pre_election: true
I20260806 20:30:12.086225 6910 raft_consensus.cc:2454] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 905e8fb9e38d4866993805f1fa379455 in term 0.
I20260806 20:30:12.086548 7057 raft_consensus.cc:2454] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 905e8fb9e38d4866993805f1fa379455 in term 0.
I20260806 20:30:12.087139 6690 leader_election.cc:302] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6464752f0e0e4c7bbd6e9063bccb9609, 905e8fb9e38d4866993805f1fa379455; no voters:
I20260806 20:30:12.087661 7135 raft_consensus.cc:2781] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 20:30:12.087826 7135 raft_consensus.cc:482] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 20:30:12.087963 7135 raft_consensus.cc:3037] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:12.090245 7135 raft_consensus.cc:504] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:12.090929 7135 leader_election.cc:288] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [CANDIDATE]: Term 1 election: Requested vote from peers 6464752f0e0e4c7bbd6e9063bccb9609 (127.17.109.194:45609), 8016649893fd4aaabf03e1a7aba14d19 (127.17.109.195:40889)
I20260806 20:30:12.091482 6910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "276d54b1855049ecbf1687e0eee1281b" candidate_uuid: "905e8fb9e38d4866993805f1fa379455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6464752f0e0e4c7bbd6e9063bccb9609"
I20260806 20:30:12.091529 7057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "276d54b1855049ecbf1687e0eee1281b" candidate_uuid: "905e8fb9e38d4866993805f1fa379455" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8016649893fd4aaabf03e1a7aba14d19"
I20260806 20:30:12.091763 6910 raft_consensus.cc:3037] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:12.091877 7057 raft_consensus.cc:3037] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 0 FOLLOWER]: Advancing to term 1
W20260806 20:30:12.092368 7111 tablet.cc:2369] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:12.094511 6910 raft_consensus.cc:2454] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 905e8fb9e38d4866993805f1fa379455 in term 1.
I20260806 20:30:12.094691 7057 raft_consensus.cc:2454] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 905e8fb9e38d4866993805f1fa379455 in term 1.
I20260806 20:30:12.095108 6690 leader_election.cc:302] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6464752f0e0e4c7bbd6e9063bccb9609, 905e8fb9e38d4866993805f1fa379455; no voters:
I20260806 20:30:12.095476 7135 raft_consensus.cc:2781] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:12.096020 7135 raft_consensus.cc:686] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 1 LEADER]: Becoming Leader. State: Replica: 905e8fb9e38d4866993805f1fa379455, State: Running, Role: LEADER
I20260806 20:30:12.096530 7135 consensus_queue.cc:237] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } }
I20260806 20:30:12.101637 6607 catalog_manager.cc:5495] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 reported cstate change: term changed from 0 to 1, leader changed from <none> to 905e8fb9e38d4866993805f1fa379455 (127.17.109.193). New cstate: current_term: 1 leader_uuid: "905e8fb9e38d4866993805f1fa379455" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "905e8fb9e38d4866993805f1fa379455" member_type: VOTER last_known_addr { host: "127.17.109.193" port: 37273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 } health_report { overall_health: UNKNOWN } } }
W20260806 20:30:12.110025 6962 tablet.cc:2369] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 20:30:12.147176 6814 tablet.cc:2369] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:12.157256 7125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.151749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.17.109.193:37273)
0806 20:30:12.151984 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:12.151990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:12.152100 (+ 110us) server_negotiation.cc:407] Connection header received
0806 20:30:12.152353 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:12.152357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:12.152444 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:12.152600 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:12.152965 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.153636 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.155042 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.155497 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.155682 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:12.156185 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:12.156671 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:12.156857 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:12.156918 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 20:30:12.156998 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260806 20:30:12.165861 6910 raft_consensus.cc:1261] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 1 FOLLOWER]: Refusing update from remote peer 905e8fb9e38d4866993805f1fa379455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:12.165861 7057 raft_consensus.cc:1261] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 1 FOLLOWER]: Refusing update from remote peer 905e8fb9e38d4866993805f1fa379455: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:12.166889 7141 consensus_queue.cc:1035] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6464752f0e0e4c7bbd6e9063bccb9609" member_type: VOTER last_known_addr { host: "127.17.109.194" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:12.167230 7135 consensus_queue.cc:1035] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8016649893fd4aaabf03e1a7aba14d19" member_type: VOTER last_known_addr { host: "127.17.109.195" port: 40889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:12.208971 6606 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:30:12.209237 6606 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 20:30:12.212944 6606 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22b3945a1e0c4158b30d6e7088c3f300: Sending DeleteTablet for 3 replicas of tablet 276d54b1855049ecbf1687e0eee1281b
I20260806 20:30:12.214181 6884 tablet_service.cc:1514] Processing DeleteTablet for tablet 276d54b1855049ecbf1687e0eee1281b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:12 UTC) from {username='slave'} at 127.0.0.1:56880
I20260806 20:30:12.214259 7033 tablet_service.cc:1514] Processing DeleteTablet for tablet 276d54b1855049ecbf1687e0eee1281b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:12 UTC) from {username='slave'} at 127.0.0.1:43752
I20260806 20:30:12.214226 6737 tablet_service.cc:1514] Processing DeleteTablet for tablet 276d54b1855049ecbf1687e0eee1281b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 20:30:12 UTC) from {username='slave'} at 127.0.0.1:34178
I20260806 20:30:12.216584 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 6673
I20260806 20:30:12.223722 7157 tablet_replica.cc:330] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: stopping tablet replica
I20260806 20:30:12.224395 7157 raft_consensus.cc:2229] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:30:12.225095 7157 raft_consensus.cc:2258] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:12.227691 7157 ts_tablet_manager.cc:1905] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:12.232985 7156 tablet_replica.cc:330] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: stopping tablet replica
I20260806 20:30:12.233688 7155 tablet_replica.cc:330] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: stopping tablet replica
I20260806 20:30:12.234006 7156 raft_consensus.cc:2229] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 1 LEADER]: Raft consensus shutting down.
I20260806 20:30:12.234308 7155 raft_consensus.cc:2229] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 20:30:12.234936 7156 raft_consensus.cc:2258] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:12.235400 7155 raft_consensus.cc:2258] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 20:30:12.237421 7156 ts_tablet_manager.cc:1905] T 276d54b1855049ecbf1687e0eee1281b P 905e8fb9e38d4866993805f1fa379455: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:12.237782 7155 ts_tablet_manager.cc:1905] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 20:30:12.238445 7157 ts_tablet_manager.cc:1918] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:12.238618 7157 log.cc:1196] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-1/wal/wals/276d54b1855049ecbf1687e0eee1281b
I20260806 20:30:12.239187 7157 ts_tablet_manager.cc:1939] T 276d54b1855049ecbf1687e0eee1281b P 6464752f0e0e4c7bbd6e9063bccb9609: Deleting consensus metadata
I20260806 20:30:12.241173 6591 catalog_manager.cc:4841] TS 6464752f0e0e4c7bbd6e9063bccb9609 (127.17.109.194:45609): tablet 276d54b1855049ecbf1687e0eee1281b (table test-table [id=a1befc2ff39c4de49f43ae4be0273094]) successfully deleted
I20260806 20:30:12.242424 7125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.218213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.17.109.193:37273)
0806 20:30:12.218478 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:12.218487 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:12.224057 (+ 5570us) server_negotiation.cc:407] Connection header received
0806 20:30:12.224254 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:12.224263 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:12.224421 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:12.224555 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:12.226501 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.228155 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.232780 (+ 4625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.233665 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.233806 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:12.240609 (+ 6803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:12.240623 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:12.240642 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:12.241439 (+ 797us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:12.241503 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:12.241526 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:12.241611 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:12.241895 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 20:30:12.242000 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:30:12.243896 6806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:12.246819 7155 ts_tablet_manager.cc:1918] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 20:30:12.246995 7155 log.cc:1196] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Deleting WAL directory at /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786047878953513-17847-0/minicluster-data/ts-2/wal/wals/276d54b1855049ecbf1687e0eee1281b
I20260806 20:30:12.247694 7155 ts_tablet_manager.cc:1939] T 276d54b1855049ecbf1687e0eee1281b P 8016649893fd4aaabf03e1a7aba14d19: Deleting consensus metadata
I20260806 20:30:12.249747 6592 catalog_manager.cc:4841] TS 8016649893fd4aaabf03e1a7aba14d19 (127.17.109.195:40889): tablet 276d54b1855049ecbf1687e0eee1281b (table test-table [id=a1befc2ff39c4de49f43ae4be0273094]) successfully deleted
I20260806 20:30:12.345449 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 6673
W20260806 20:30:12.367828 6592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.109.193:37273 (error 108)
I20260806 20:30:12.367973 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 6817
I20260806 20:30:12.369925 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.369394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37273 (local address 127.0.0.1:34200)
0806 20:30:12.369601 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.369743 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:37273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260806 20:30:12.370317 6592 catalog_manager.cc:4584] TS 905e8fb9e38d4866993805f1fa379455 (127.17.109.193:37273): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 276d54b1855049ecbf1687e0eee1281b: Network error: Client connection negotiation failed: client connection to 127.17.109.193:37273: connect: Connection refused (error 111)
I20260806 20:30:12.375653 7124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.368777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56892 (local address 127.17.109.194:45609)
0806 20:30:12.368954 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:12.368960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:12.369266 (+ 306us) server_negotiation.cc:407] Connection header received
0806 20:30:12.369333 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:12.369337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:12.369407 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:12.369548 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:12.370212 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.371118 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.372490 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.372943 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.373085 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:12.374211 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:12.374219 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:12.374232 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:12.374802 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:12.374857 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:12.374875 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:12.375013 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:12.375151 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 20:30:12.375254 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 20:30:12.376330 6955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:12.423962 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.423401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37273 (local address 127.0.0.1:34214)
0806 20:30:12.423696 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.423802 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:37273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 20:30:12.425107 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.424781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37273 (local address 127.0.0.1:34226)
0806 20:30:12.424919 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.424991 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:37273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 20:30:12.469331 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 6817
I20260806 20:30:12.485895 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.485375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37273 (local address 127.0.0.1:34238)
0806 20:30:12.485609 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.485701 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:37273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260806 20:30:12.486975 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.486619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37273 (local address 127.0.0.1:34248)
0806 20:30:12.486778 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.486859 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:37273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 20:30:12.489645 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 6964
I20260806 20:30:12.496578 7127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.490448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.17.109.195:40889)
0806 20:30:12.490637 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:12.490642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:12.490662 (+ 20us) server_negotiation.cc:407] Connection header received
0806 20:30:12.490738 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:12.490743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:12.490836 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:12.490918 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:12.491298 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.492131 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.493399 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:12.493996 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:12.494096 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:12.495057 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:12.495065 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:12.495080 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:12.495830 (+ 750us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:12.495880 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:12.495900 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:12.495988 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:12.496121 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 20:30:12.496207 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:30:12.497247 7105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:12.568030 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.567406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37273 (local address 127.0.0.1:34252)
0806 20:30:12.567725 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.567851 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:37273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260806 20:30:12.569248 7126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:12.568842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.193:37273 (local address 127.0.0.1:34256)
0806 20:30:12.569053 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:12.569130 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.193:37273: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260806 20:30:12.588639 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 6964
I20260806 20:30:12.608996 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 6574
I20260806 20:30:12.610249 6641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:12.722203 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 6574
2026-08-06T20:30:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2986 ms)
[----------] 6 tests from EncryptionPolicyTest (23517 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:12 dist-test-slave-6k22 krb5kdc[7166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:12 dist-test-slave-6k22 krb5kdc[7166](info): set up 2 sockets
Aug 06 20:30:12 dist-test-slave-6k22 krb5kdc[7166](info): commencing operation
krb5kdc: starting...
W20260806 20:30:13.935897 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:14 dist-test-slave-6k22 krb5kdc[7166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:14.275503 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:14.627915 7182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:14.628253 7182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:14.628444 7182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:14.635833 7182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:14.635958 7182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:14.636005 7182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:14.636104 7182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:14.636170 7182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:14.646373 7182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:14.647863 7182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:14.649276 7182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:14.661603 7188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:14.661690 7189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:14.661657 7191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:14.662742 7182 server_base.cc:1034] running on GCE node
I20260806 20:30:14.663681 7182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:14.665144 7182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:14.666378 7182 hybrid_clock.cc:648] HybridClock initialized: now 1786048214666294 us; error 67 us; skew 500 ppm
I20260806 20:30:14.666765 7182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:14 dist-test-slave-6k22 krb5kdc[7166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:14.670171 7182 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:14.671852 7182 webserver.cc:460] Webserver started at http://127.0.0.1:35617/ using document root <none> and password file <none>
I20260806 20:30:14.672538 7182 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:14.672639 7182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:14.672945 7182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:14.675988 7182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "d5598ecca4d944d999bc218dc8d04f37"
format_stamp: "Formatted at 2026-08-06 20:30:14 on dist-test-slave-6k22"
server_key: "ab92811da321c6b915058d59ad0b631a"
server_key_iv: "26ffb4d1921f145a014032cf7c9feaaf"
server_key_version: "encryptionkey@0"
I20260806 20:30:14.676756 7182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "d5598ecca4d944d999bc218dc8d04f37"
format_stamp: "Formatted at 2026-08-06 20:30:14 on dist-test-slave-6k22"
server_key: "ab92811da321c6b915058d59ad0b631a"
server_key_iv: "26ffb4d1921f145a014032cf7c9feaaf"
server_key_version: "encryptionkey@0"
I20260806 20:30:14.681936 7182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260806 20:30:14.685527 7198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:14.686847 7182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260806 20:30:14.687042 7182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "d5598ecca4d944d999bc218dc8d04f37"
format_stamp: "Formatted at 2026-08-06 20:30:14 on dist-test-slave-6k22"
server_key: "ab92811da321c6b915058d59ad0b631a"
server_key_iv: "26ffb4d1921f145a014032cf7c9feaaf"
server_key_version: "encryptionkey@0"
I20260806 20:30:14.687263 7182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:14.717995 7182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:14.722616 7182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:14.722975 7182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:14.746007 7182 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43037
I20260806 20:30:14.746057 7258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43037 every 8 connection(s)
I20260806 20:30:14.747486 7182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:14.750085 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 7182
I20260806 20:30:14.750341 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:14.751065 17847 external_mini_cluster.cc:1428] Setting key 81b8ab37890bec933f2fa77387214930
I20260806 20:30:14.751539 7259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:14.766685 7259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37: Bootstrap starting.
Aug 06 20:30:14 dist-test-slave-6k22 krb5kdc[7166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 20:30:14.770843 7259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:14.772573 7259 log.cc:824] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:14.776816 7259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37: No bootstrap required, opened a new log
I20260806 20:30:14.778699 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:14.753247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45733 (local address 127.0.0.1:43037)
0806 20:30:14.754123 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:14.754134 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:14.754178 (+ 44us) server_negotiation.cc:407] Connection header received
0806 20:30:14.756282 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:14.756305 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:14.756726 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:14.757068 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:14.759014 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:14.760396 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:14.762266 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:14.762989 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:14.763130 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:14.768352 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:14.768369 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:14.768392 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:14.768461 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:14.773046 (+ 4585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:14.774243 (+ 1197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:14.774253 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:14.774263 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:14.774506 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:14.775027 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:14.775032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:14.775037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:14.775865 (+ 828us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:14.776269 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:14.776673 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 20:30:14.777066 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":388,"threads_started":1}
I20260806 20:30:14.783576 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:14.790093 7259 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5598ecca4d944d999bc218dc8d04f37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43037 } }
I20260806 20:30:14.790733 7259 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:14.790912 7259 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5598ecca4d944d999bc218dc8d04f37, State: Initialized, Role: FOLLOWER
I20260806 20:30:14.791745 7259 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5598ecca4d944d999bc218dc8d04f37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43037 } }
I20260806 20:30:14.792089 7259 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:14.792263 7259 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:14.792513 7259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:14.796205 7259 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5598ecca4d944d999bc218dc8d04f37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43037 } }
I20260806 20:30:14.796965 7259 leader_election.cc:302] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5598ecca4d944d999bc218dc8d04f37; no voters:
I20260806 20:30:14.797852 7259 leader_election.cc:288] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:14.797998 7270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:14.799364 7270 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [term 1 LEADER]: Becoming Leader. State: Replica: d5598ecca4d944d999bc218dc8d04f37, State: Running, Role: LEADER
I20260806 20:30:14.800045 7270 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5598ecca4d944d999bc218dc8d04f37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43037 } }
I20260806 20:30:14.800830 7259 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:14.805229 7271 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5598ecca4d944d999bc218dc8d04f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5598ecca4d944d999bc218dc8d04f37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43037 } } }
I20260806 20:30:14.805255 7272 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5598ecca4d944d999bc218dc8d04f37. Latest consensus state: current_term: 1 leader_uuid: "d5598ecca4d944d999bc218dc8d04f37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5598ecca4d944d999bc218dc8d04f37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43037 } } }
I20260806 20:30:14.805727 7271 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:14.805804 7272 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:14.813076 7263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:14.790496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51043 (local address 127.0.0.1:43037)
0806 20:30:14.790735 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:14.790742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:14.791185 (+ 443us) server_negotiation.cc:407] Connection header received
0806 20:30:14.791345 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:14.791352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:14.791450 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:14.791620 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:14.793167 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:14.794579 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:14.795883 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:14.796733 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:14.796889 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:14.800341 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:14.800350 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:14.800355 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:14.800412 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:14.805900 (+ 5488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:14.807735 (+ 1835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:14.807746 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:14.807753 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:14.808027 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:14.808967 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:14.808977 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:14.808984 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:14.809671 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:14.811846 (+ 2175us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:14.812145 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 20:30:14.812419 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 20:30:14.819933 7279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:14.827186 7279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:14.837625 7279 catalog_manager.cc:1347] Generated new cluster ID: 5c233d4ac81f486aadec166bb6db45c3
I20260806 20:30:14.837774 7279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:14.863698 7279 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:14.865420 7279 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:14.878048 7279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5598ecca4d944d999bc218dc8d04f37: Generated new TSK 0
I20260806 20:30:14.879042 7279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:14.893201 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7182
I20260806 20:30:14.895130 7252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:14.995913 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 7182
W20260806 20:30:15.017714 7164 connection.cc:535] client connection to 127.0.0.1:43037 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43037): Connection reset by peer (error 104)
2026-08-06T20:30:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2262 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:15 dist-test-slave-6k22 krb5kdc[7294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:15 dist-test-slave-6k22 krb5kdc[7294](info): set up 2 sockets
Aug 06 20:30:15 dist-test-slave-6k22 krb5kdc[7294](info): commencing operation
krb5kdc: starting...
W20260806 20:30:16.228837 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:16 dist-test-slave-6k22 krb5kdc[7294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:16.571486 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39881
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:16.918672 7307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:16.918959 7307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:16.919054 7307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:16.926374 7307 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:16.926491 7307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:16.926540 7307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:16.926614 7307 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:16.926659 7307 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:16.936664 7307 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39881
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:16.938016 7307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:16.939327 7307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:16.951304 7315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:16.951357 7313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:16.951596 7312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:16.952421 7307 server_base.cc:1034] running on GCE node
I20260806 20:30:16.953277 7307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:16.955016 7307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:16.956290 7307 hybrid_clock.cc:648] HybridClock initialized: now 1786048216956233 us; error 48 us; skew 500 ppm
I20260806 20:30:16.956707 7307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:16 dist-test-slave-6k22 krb5kdc[7294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:16.960001 7307 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:16.961542 7307 webserver.cc:460] Webserver started at http://127.0.0.1:42477/ using document root <none> and password file <none>
I20260806 20:30:16.962255 7307 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:16.962412 7307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:16.962741 7307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:16.966005 7307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b"
format_stamp: "Formatted at 2026-08-06 20:30:16 on dist-test-slave-6k22"
server_key: "7a0d808909f8aea78edd110b67fe437b"
server_key_iv: "a7681f34f33079a7b14ff7f827453a3e"
server_key_version: "encryptionkey@0"
I20260806 20:30:16.966856 7307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b"
format_stamp: "Formatted at 2026-08-06 20:30:16 on dist-test-slave-6k22"
server_key: "7a0d808909f8aea78edd110b67fe437b"
server_key_iv: "a7681f34f33079a7b14ff7f827453a3e"
server_key_version: "encryptionkey@0"
I20260806 20:30:16.972587 7307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 20:30:16.976554 7323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:16.978021 7307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:30:16.978240 7307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b"
format_stamp: "Formatted at 2026-08-06 20:30:16 on dist-test-slave-6k22"
server_key: "7a0d808909f8aea78edd110b67fe437b"
server_key_iv: "a7681f34f33079a7b14ff7f827453a3e"
server_key_version: "encryptionkey@0"
I20260806 20:30:16.978492 7307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:17.005856 7307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:17.010820 7307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:17.011189 7307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:17.030836 7307 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39881
I20260806 20:30:17.030855 7380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39881 every 8 connection(s)
I20260806 20:30:17.032335 7307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:17.035497 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 7307
I20260806 20:30:17.035748 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:17.036219 17847 external_mini_cluster.cc:1428] Setting key 5027aaa323d2848da4f73b214dd46951
I20260806 20:30:17.036358 7381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:17.050709 7381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b: Bootstrap starting.
I20260806 20:30:17.051235 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:17.038118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35081 (local address 127.0.0.1:39881)
0806 20:30:17.038982 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:17.038994 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:17.039041 (+ 47us) server_negotiation.cc:407] Connection header received
0806 20:30:17.041221 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:17.041243 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:17.041656 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:17.042047 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:17.042674 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:17.044165 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:17.046078 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:17.046758 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:17.046942 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:17.048480 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:17.048490 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:17.048513 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:17.049166 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:17.049259 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:17.049290 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:17.049441 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:17.049898 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 20:30:17.050105 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":417,"threads_started":1}
I20260806 20:30:17.054558 7381 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:17.055423 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:17.055847 7381 log.cc:824] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:17.058902 7381 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b: No bootstrap required, opened a new log
I20260806 20:30:17.067616 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:17.060462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49459 (local address 127.0.0.1:39881)
0806 20:30:17.060666 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:17.060672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:17.061177 (+ 505us) server_negotiation.cc:407] Connection header received
0806 20:30:17.061345 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:17.061356 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:17.061437 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:17.061561 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:17.062127 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:17.063292 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:17.064689 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:17.065425 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:17.065564 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:17.066753 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:17.066760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:17.066766 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:17.066792 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:17.066844 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:17.066866 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:17.066996 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:17.067177 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 20:30:17.067239 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 20:30:17.068564 7381 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39881 } }
I20260806 20:30:17.068984 7381 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:17.069118 7381 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b374a1d3e3a4d66baae7c4e1d0c854b, State: Initialized, Role: FOLLOWER
I20260806 20:30:17.069693 7381 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39881 } }
I20260806 20:30:17.069981 7381 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:17.070150 7381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:17.070338 7381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:17.072700 7381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39881 } }
I20260806 20:30:17.073240 7381 leader_election.cc:302] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b374a1d3e3a4d66baae7c4e1d0c854b; no voters:
I20260806 20:30:17.073904 7381 leader_election.cc:288] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:17.074060 7392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:17.075237 7392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [term 1 LEADER]: Becoming Leader. State: Replica: 6b374a1d3e3a4d66baae7c4e1d0c854b, State: Running, Role: LEADER
I20260806 20:30:17.075838 7392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39881 } }
I20260806 20:30:17.076356 7381 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:17.078861 7393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39881 } } }
I20260806 20:30:17.079375 7393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:17.078950 7394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b374a1d3e3a4d66baae7c4e1d0c854b. Latest consensus state: current_term: 1 leader_uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b374a1d3e3a4d66baae7c4e1d0c854b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39881 } } }
I20260806 20:30:17.079792 7394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:17.084591 7403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:17.089932 7403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:17.099073 7403 catalog_manager.cc:1347] Generated new cluster ID: 4083d46f0b3e4aeea7cac0e1bce5c2cf
I20260806 20:30:17.099211 7403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:17.153416 7403 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:17.154963 7403 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:17.162569 7403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b374a1d3e3a4d66baae7c4e1d0c854b: Generated new TSK 0
I20260806 20:30:17.163643 7403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:17.169620 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7307
W20260806 20:30:17.171145 7375 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35081
W20260806 20:30:17.172046 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7307: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:17.172274 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 7307
W20260806 20:30:17.193096 7292 connection.cc:535] client connection to 127.0.0.1:39881 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39881): Connection reset by peer (error 104)
2026-08-06T20:30:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2174 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:17 dist-test-slave-6k22 krb5kdc[7417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:17 dist-test-slave-6k22 krb5kdc[7417](info): set up 2 sockets
Aug 06 20:30:17 dist-test-slave-6k22 krb5kdc[7417](info): commencing operation
krb5kdc: starting...
W20260806 20:30:18.377135 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:18 dist-test-slave-6k22 krb5kdc[7417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:18.731904 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:19.099503 7431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:19.099992 7431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:19.100194 7431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:19.108188 7431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:19.108330 7431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:19.108466 7431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:19.108603 7431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:19.108704 7431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:19.120275 7431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42299
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:19.121901 7431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:19.123337 7431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:19.136361 7437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:19.136283 7436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:19.136839 7440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:19.137163 7431 server_base.cc:1034] running on GCE node
I20260806 20:30:19.137871 7431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:19.139321 7431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:19.140594 7431 hybrid_clock.cc:648] HybridClock initialized: now 1786048219140518 us; error 55 us; skew 500 ppm
I20260806 20:30:19.141003 7431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:19 dist-test-slave-6k22 krb5kdc[7417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048219, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:19.144366 7431 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:19.145967 7431 webserver.cc:460] Webserver started at http://127.0.0.1:33203/ using document root <none> and password file <none>
I20260806 20:30:19.146663 7431 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:19.146857 7431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:19.147194 7431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:19.150269 7431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "8a5687bdb11b40dc852ac998df408a40"
format_stamp: "Formatted at 2026-08-06 20:30:19 on dist-test-slave-6k22"
server_key: "b10c1be24798e0381688f2657c50238e"
server_key_iv: "cded37323630a6330a0c67b6ed1197ec"
server_key_version: "encryptionkey@0"
I20260806 20:30:19.151068 7431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "8a5687bdb11b40dc852ac998df408a40"
format_stamp: "Formatted at 2026-08-06 20:30:19 on dist-test-slave-6k22"
server_key: "b10c1be24798e0381688f2657c50238e"
server_key_iv: "cded37323630a6330a0c67b6ed1197ec"
server_key_version: "encryptionkey@0"
I20260806 20:30:19.156312 7431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 20:30:19.159934 7448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:19.161254 7431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 20:30:19.161461 7431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "8a5687bdb11b40dc852ac998df408a40"
format_stamp: "Formatted at 2026-08-06 20:30:19 on dist-test-slave-6k22"
server_key: "b10c1be24798e0381688f2657c50238e"
server_key_iv: "cded37323630a6330a0c67b6ed1197ec"
server_key_version: "encryptionkey@0"
I20260806 20:30:19.161676 7431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:19.167388 7431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:19.171886 7431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:19.172197 7431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:19.191849 7431 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42299
I20260806 20:30:19.191887 7502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42299 every 8 connection(s)
I20260806 20:30:19.193317 7431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:19.197090 7503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:19.197696 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 7431
I20260806 20:30:19.197925 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:19.198406 17847 external_mini_cluster.cc:1428] Setting key 9b2631c86db2ca123ca2d84f567a09a4
I20260806 20:30:19.206584 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:19.200237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38017 (local address 127.0.0.1:42299)
0806 20:30:19.201107 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:19.201119 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:19.201159 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:30:19.203081 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:19.203106 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:19.203449 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:19.203892 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:19.205014 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:19.205021 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:19.205034 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:19.205048 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:19.205114 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:19.205134 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:19.205316 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:19.205423 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 20:30:19.205650 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":396,"threads_started":1}
I20260806 20:30:19.210140 7503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40: Bootstrap starting.
I20260806 20:30:19.210450 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:19.214085 7503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:19.215490 7503 log.cc:824] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:19.218708 7506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:19.215895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47221 (local address 127.0.0.1:42299)
0806 20:30:19.216134 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:19.216140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:19.216559 (+ 419us) server_negotiation.cc:407] Connection header received
0806 20:30:19.216727 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:19.216733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:19.216797 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:19.216927 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:19.217890 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:19.217895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:19.217902 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:19.217919 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:19.217964 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:19.217981 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:19.218104 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:19.218258 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 20:30:19.218314 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:30:19.219288 7503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40: No bootstrap required, opened a new log
I20260806 20:30:19.227223 7503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a5687bdb11b40dc852ac998df408a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } }
I20260806 20:30:19.227646 7503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:19.227747 7503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a5687bdb11b40dc852ac998df408a40, State: Initialized, Role: FOLLOWER
I20260806 20:30:19.228413 7503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a5687bdb11b40dc852ac998df408a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } }
I20260806 20:30:19.228658 7503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:19.228813 7503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:19.228987 7503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:19.231223 7503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a5687bdb11b40dc852ac998df408a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } }
I20260806 20:30:19.231789 7503 leader_election.cc:302] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a5687bdb11b40dc852ac998df408a40; no voters:
I20260806 20:30:19.232389 7503 leader_election.cc:288] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:19.232524 7513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:19.233234 7513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [term 1 LEADER]: Becoming Leader. State: Replica: 8a5687bdb11b40dc852ac998df408a40, State: Running, Role: LEADER
I20260806 20:30:19.233870 7513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a5687bdb11b40dc852ac998df408a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } }
I20260806 20:30:19.234561 7503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:19.236719 7514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a5687bdb11b40dc852ac998df408a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a5687bdb11b40dc852ac998df408a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } } }
I20260806 20:30:19.236840 7515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a5687bdb11b40dc852ac998df408a40. Latest consensus state: current_term: 1 leader_uuid: "8a5687bdb11b40dc852ac998df408a40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a5687bdb11b40dc852ac998df408a40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42299 } } }
I20260806 20:30:19.237164 7515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:19.237164 7514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:19.239370 7519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:19.244639 7519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:19.253863 7519 catalog_manager.cc:1347] Generated new cluster ID: dae6ea1549f249b6a215be56b5f03fd8
I20260806 20:30:19.253986 7519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:19.281512 7519 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:19.282042 7519 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:19.301946 7519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a5687bdb11b40dc852ac998df408a40: Generated new TSK 0
I20260806 20:30:19.303052 7519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:19.316900 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7431
W20260806 20:30:19.318055 7498 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38017
W20260806 20:30:19.318837 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7431: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:19.318997 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 7431
2026-08-06T20:30:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2146 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:19 dist-test-slave-6k22 krb5kdc[7535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:19 dist-test-slave-6k22 krb5kdc[7535](info): set up 2 sockets
Aug 06 20:30:19 dist-test-slave-6k22 krb5kdc[7535](info): commencing operation
krb5kdc: starting...
W20260806 20:30:20.605355 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.002s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:20 dist-test-slave-6k22 krb5kdc[7535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:20.951352 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:21.300971 7549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:21.301290 7549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:21.301465 7549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:21.308892 7549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:21.309028 7549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:21.309152 7549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:21.309273 7549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:21.309365 7549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:21.319474 7549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:21.320950 7549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:21.322322 7549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:21.334800 7556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:21.334820 7559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:21.334863 7557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:21.335886 7549 server_base.cc:1034] running on GCE node
I20260806 20:30:21.336717 7549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:21.338197 7549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:21.339406 7549 hybrid_clock.cc:648] HybridClock initialized: now 1786048221339372 us; error 39 us; skew 500 ppm
I20260806 20:30:21.339809 7549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:21 dist-test-slave-6k22 krb5kdc[7535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048221, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:21.342969 7549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:21.344601 7549 webserver.cc:460] Webserver started at http://127.0.0.1:39795/ using document root <none> and password file <none>
I20260806 20:30:21.345284 7549 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:21.345441 7549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:21.345753 7549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:21.348901 7549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "0596755c82594d29b1d9afe019d4b9d3"
format_stamp: "Formatted at 2026-08-06 20:30:21 on dist-test-slave-6k22"
server_key: "149affdb6842e3460a43277e59f6ce8b"
server_key_iv: "78fe64d7fda2e40c0cc2e70e0c205c2c"
server_key_version: "encryptionkey@0"
I20260806 20:30:21.349690 7549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "0596755c82594d29b1d9afe019d4b9d3"
format_stamp: "Formatted at 2026-08-06 20:30:21 on dist-test-slave-6k22"
server_key: "149affdb6842e3460a43277e59f6ce8b"
server_key_iv: "78fe64d7fda2e40c0cc2e70e0c205c2c"
server_key_version: "encryptionkey@0"
I20260806 20:30:21.354930 7549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:30:21.358500 7567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:21.359956 7549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:30:21.360140 7549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "0596755c82594d29b1d9afe019d4b9d3"
format_stamp: "Formatted at 2026-08-06 20:30:21 on dist-test-slave-6k22"
server_key: "149affdb6842e3460a43277e59f6ce8b"
server_key_iv: "78fe64d7fda2e40c0cc2e70e0c205c2c"
server_key_version: "encryptionkey@0"
I20260806 20:30:21.360337 7549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:21.390751 7549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:21.395015 7549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:21.395356 7549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:21.414443 7549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44715
I20260806 20:30:21.414466 7621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44715 every 8 connection(s)
I20260806 20:30:21.416019 7549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:21.419998 7623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:21.425556 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 7549
I20260806 20:30:21.425747 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:21.426362 17847 external_mini_cluster.cc:1428] Setting key 3eb0d5f14268c96c20690d5473dce4a1
I20260806 20:30:21.432037 7623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3: Bootstrap starting.
I20260806 20:30:21.435964 7623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:21.437152 7623 log.cc:824] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:21.441826 7623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3: No bootstrap required, opened a new log
Aug 06 20:30:21 dist-test-slave-6k22 krb5kdc[7535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 20:30:21.450510 7623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0596755c82594d29b1d9afe019d4b9d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } }
I20260806 20:30:21.450930 7623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:21.451083 7623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0596755c82594d29b1d9afe019d4b9d3, State: Initialized, Role: FOLLOWER
I20260806 20:30:21.451932 7623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0596755c82594d29b1d9afe019d4b9d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } }
I20260806 20:30:21.452224 7623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:21.452407 7623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:21.452598 7623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:21.452924 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:21.428458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52193 (local address 127.0.0.1:44715)
0806 20:30:21.429334 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:21.429353 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:21.429405 (+ 52us) server_negotiation.cc:407] Connection header received
0806 20:30:21.431832 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:21.431864 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:21.432366 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:21.432799 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:21.434232 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:21.436357 (+ 2125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:21.438192 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:21.439447 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:21.439677 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:21.444815 (+ 5138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:21.444825 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:21.444842 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:21.444885 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:21.448417 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:21.449167 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:21.449174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:21.449181 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:21.449358 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:21.449788 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:21.449795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:21.449798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:21.450338 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:21.450642 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:21.451121 (+ 479us) server_negotiation.cc:299] Negotiation successful
0806 20:30:21.451630 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":387,"threads_started":1}
I20260806 20:30:21.456166 7623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0596755c82594d29b1d9afe019d4b9d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } }
I20260806 20:30:21.456769 7623 leader_election.cc:302] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0596755c82594d29b1d9afe019d4b9d3; no voters:
I20260806 20:30:21.457500 7623 leader_election.cc:288] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:21.457545 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:21.457628 7629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:21.459826 7629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [term 1 LEADER]: Becoming Leader. State: Replica: 0596755c82594d29b1d9afe019d4b9d3, State: Running, Role: LEADER
I20260806 20:30:21.460126 7623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:21.460446 7629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0596755c82594d29b1d9afe019d4b9d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } }
I20260806 20:30:21.463806 7630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0596755c82594d29b1d9afe019d4b9d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0596755c82594d29b1d9afe019d4b9d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } } }
I20260806 20:30:21.464286 7630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:21.464949 7631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0596755c82594d29b1d9afe019d4b9d3. Latest consensus state: current_term: 1 leader_uuid: "0596755c82594d29b1d9afe019d4b9d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0596755c82594d29b1d9afe019d4b9d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44715 } } }
I20260806 20:30:21.465307 7631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:21.487447 7639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:21.495362 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:21.463221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37327 (local address 127.0.0.1:44715)
0806 20:30:21.463390 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:21.463399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:21.474233 (+ 10834us) server_negotiation.cc:407] Connection header received
0806 20:30:21.474913 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:21.474924 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:21.475042 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:21.475137 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:21.478090 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:21.479661 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:21.482216 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:21.483212 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:21.483343 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:21.487892 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:21.487903 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:21.487908 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:21.487976 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:21.492306 (+ 4330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:21.493043 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:21.493049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:21.493053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:21.493204 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:21.493642 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:21.493648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:21.493659 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:21.493980 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:21.494243 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:21.494634 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 20:30:21.494901 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 20:30:21.503288 7639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:21.513180 7639 catalog_manager.cc:1347] Generated new cluster ID: 42b932ac2c0649e9a210eb8ba635ad2f
I20260806 20:30:21.513334 7639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:21.526734 7639 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:21.528662 7639 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:21.548750 7639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0596755c82594d29b1d9afe019d4b9d3: Generated new TSK 0
I20260806 20:30:21.549857 7639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:21.558037 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7549
I20260806 20:30:21.559264 7617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:21.654135 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 7549
2026-08-06T20:30:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:21 dist-test-slave-6k22 krb5kdc[7659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:21 dist-test-slave-6k22 krb5kdc[7659](info): set up 2 sockets
Aug 06 20:30:21 dist-test-slave-6k22 krb5kdc[7659](info): commencing operation
krb5kdc: starting...
W20260806 20:30:22.855806 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.006s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:23 dist-test-slave-6k22 krb5kdc[7659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:23.215025 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:23.562947 7674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:23.563313 7674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:23.563495 7674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:23.570909 7674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:23.571082 7674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:23.571199 7674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:23.571316 7674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:23.571409 7674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:23.581521 7674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:23.583047 7674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:23.584508 7674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:23.596905 7680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:23.596938 7679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:23.597005 7682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:23.597697 7674 server_base.cc:1034] running on GCE node
I20260806 20:30:23.598592 7674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:23.600238 7674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:23.601449 7674 hybrid_clock.cc:648] HybridClock initialized: now 1786048223601405 us; error 39 us; skew 500 ppm
I20260806 20:30:23.601850 7674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:23 dist-test-slave-6k22 krb5kdc[7659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048223, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:23.614259 7674 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:23.616083 7674 webserver.cc:460] Webserver started at http://127.0.0.1:34561/ using document root <none> and password file <none>
I20260806 20:30:23.616778 7674 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:23.616895 7674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:23.617161 7674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:23.620379 7674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "c7e6319c81644eeba206eba376bd0477"
format_stamp: "Formatted at 2026-08-06 20:30:23 on dist-test-slave-6k22"
server_key: "29184b6e083430fc578c7d0d5f351f4d"
server_key_iv: "7a13ec8bc6dc91cde371f1c402cda8c7"
server_key_version: "encryptionkey@0"
I20260806 20:30:23.621112 7674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "c7e6319c81644eeba206eba376bd0477"
format_stamp: "Formatted at 2026-08-06 20:30:23 on dist-test-slave-6k22"
server_key: "29184b6e083430fc578c7d0d5f351f4d"
server_key_iv: "7a13ec8bc6dc91cde371f1c402cda8c7"
server_key_version: "encryptionkey@0"
I20260806 20:30:23.626453 7674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:30:23.630002 7691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:23.631385 7674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:30:23.631656 7674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "c7e6319c81644eeba206eba376bd0477"
format_stamp: "Formatted at 2026-08-06 20:30:23 on dist-test-slave-6k22"
server_key: "29184b6e083430fc578c7d0d5f351f4d"
server_key_iv: "7a13ec8bc6dc91cde371f1c402cda8c7"
server_key_version: "encryptionkey@0"
I20260806 20:30:23.631914 7674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:23.647204 7674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:23.651369 7674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:23.651726 7674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:23.672068 7674 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39021
I20260806 20:30:23.672174 7746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39021 every 8 connection(s)
I20260806 20:30:23.673780 7674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:23.677800 7747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:23.679185 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 7674
I20260806 20:30:23.679406 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:23.679993 17847 external_mini_cluster.cc:1428] Setting key 03326144221e1ad67da65727751f3567
I20260806 20:30:23.691995 7747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477: Bootstrap starting.
I20260806 20:30:23.693809 7750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:23.681983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40063 (local address 127.0.0.1:39021)
0806 20:30:23.682956 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:23.682968 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:23.683020 (+ 52us) server_negotiation.cc:407] Connection header received
0806 20:30:23.684756 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:23.684781 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:23.685198 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:23.685607 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:23.686135 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:23.687419 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:23.688780 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:23.689891 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:23.690086 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:23.691379 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:23.691386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:23.691400 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:23.692017 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:23.692127 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:23.692162 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:23.692256 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:23.692495 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 20:30:23.692799 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":532,"threads_started":1}
I20260806 20:30:23.695802 7747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:23.697130 7747 log.cc:824] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:23.697845 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:23.700454 7747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477: No bootstrap required, opened a new log
I20260806 20:30:23.710043 7750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:23.703420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40711 (local address 127.0.0.1:39021)
0806 20:30:23.703645 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:23.703656 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:23.704038 (+ 382us) server_negotiation.cc:407] Connection header received
0806 20:30:23.704180 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:23.704185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:23.704305 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:23.704422 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:23.704873 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:23.705984 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:23.707302 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:23.708068 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:23.708184 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:23.709271 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:23.709278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:23.709285 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:23.709308 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:23.709363 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:23.709384 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:23.709476 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:23.709612 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 20:30:23.709708 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:30:23.710773 7747 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e6319c81644eeba206eba376bd0477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } }
I20260806 20:30:23.711167 7747 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:23.711298 7747 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e6319c81644eeba206eba376bd0477, State: Initialized, Role: FOLLOWER
I20260806 20:30:23.711966 7747 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e6319c81644eeba206eba376bd0477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } }
I20260806 20:30:23.712214 7747 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:23.712383 7747 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:23.712561 7747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:23.714840 7747 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e6319c81644eeba206eba376bd0477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } }
I20260806 20:30:23.715371 7747 leader_election.cc:302] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7e6319c81644eeba206eba376bd0477; no voters:
I20260806 20:30:23.716111 7747 leader_election.cc:288] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:23.716310 7757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:23.717018 7757 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [term 1 LEADER]: Becoming Leader. State: Replica: c7e6319c81644eeba206eba376bd0477, State: Running, Role: LEADER
I20260806 20:30:23.717664 7757 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e6319c81644eeba206eba376bd0477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } }
I20260806 20:30:23.718566 7747 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:23.720949 7759 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7e6319c81644eeba206eba376bd0477. Latest consensus state: current_term: 1 leader_uuid: "c7e6319c81644eeba206eba376bd0477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e6319c81644eeba206eba376bd0477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } } }
I20260806 20:30:23.721143 7758 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7e6319c81644eeba206eba376bd0477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e6319c81644eeba206eba376bd0477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39021 } } }
I20260806 20:30:23.721451 7759 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:23.721475 7758 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:23.723690 7767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:23.728629 7767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:23.737396 7767 catalog_manager.cc:1347] Generated new cluster ID: 028a79d193f44fb8ba30b65667f8906a
I20260806 20:30:23.737571 7767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:23.761410 7767 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:23.762780 7767 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:23.773337 7767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7e6319c81644eeba206eba376bd0477: Generated new TSK 0
I20260806 20:30:23.774169 7767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:23.781435 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7674
W20260806 20:30:23.782521 7742 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40063
W20260806 20:30:23.783385 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7674: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:23.783545 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 7674
2026-08-06T20:30:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2128 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:23 dist-test-slave-6k22 krb5kdc[7780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:23 dist-test-slave-6k22 krb5kdc[7780](info): set up 2 sockets
Aug 06 20:30:23 dist-test-slave-6k22 krb5kdc[7780](info): commencing operation
krb5kdc: starting...
W20260806 20:30:25.015954 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:25 dist-test-slave-6k22 krb5kdc[7780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:25.351392 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:25.697861 7794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:25.698145 7794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:25.698302 7794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:25.705569 7794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:25.705687 7794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:25.705796 7794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:25.705912 7794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:25.705994 7794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:25.716990 7794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:25.718415 7794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:25.719843 7794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:25.733038 7802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:25.733160 7800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:25.733042 7799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:25.737095 7794 server_base.cc:1034] running on GCE node
I20260806 20:30:25.737931 7794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:25.739450 7794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:25.740696 7794 hybrid_clock.cc:648] HybridClock initialized: now 1786048225740636 us; error 54 us; skew 500 ppm
I20260806 20:30:25.741075 7794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:25 dist-test-slave-6k22 krb5kdc[7780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:25.744546 7794 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:25.746110 7794 webserver.cc:460] Webserver started at http://127.0.0.1:34825/ using document root <none> and password file <none>
I20260806 20:30:25.746891 7794 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:25.746999 7794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:25.747259 7794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:25.750342 7794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "fb8d1f0682b44138afc179fa6f131394"
format_stamp: "Formatted at 2026-08-06 20:30:25 on dist-test-slave-6k22"
server_key: "0e3dd02f7c1110fcae6335abc736fa21"
server_key_iv: "1bd11bd63a85b072420c4bb84239f0ad"
server_key_version: "encryptionkey@0"
I20260806 20:30:25.751080 7794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "fb8d1f0682b44138afc179fa6f131394"
format_stamp: "Formatted at 2026-08-06 20:30:25 on dist-test-slave-6k22"
server_key: "0e3dd02f7c1110fcae6335abc736fa21"
server_key_iv: "1bd11bd63a85b072420c4bb84239f0ad"
server_key_version: "encryptionkey@0"
I20260806 20:30:25.756464 7794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260806 20:30:25.760131 7810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:25.761579 7794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 20:30:25.761770 7794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "fb8d1f0682b44138afc179fa6f131394"
format_stamp: "Formatted at 2026-08-06 20:30:25 on dist-test-slave-6k22"
server_key: "0e3dd02f7c1110fcae6335abc736fa21"
server_key_iv: "1bd11bd63a85b072420c4bb84239f0ad"
server_key_version: "encryptionkey@0"
I20260806 20:30:25.761950 7794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:25.768410 7794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:25.772890 7794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:25.773162 7794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:25.793780 7794 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41777
I20260806 20:30:25.793859 7866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41777 every 8 connection(s)
I20260806 20:30:25.795396 7794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:25.799103 7867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:25.804407 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 7794
I20260806 20:30:25.804621 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:25.805171 17847 external_mini_cluster.cc:1428] Setting key 2417fa05563b3ad684491f81ed1cd00b
I20260806 20:30:25.811259 7867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394: Bootstrap starting.
I20260806 20:30:25.814647 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:25.807236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36187 (local address 127.0.0.1:41777)
0806 20:30:25.808212 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:25.808240 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:25.808289 (+ 49us) server_negotiation.cc:407] Connection header received
0806 20:30:25.810645 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:25.810675 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:25.811102 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:25.811535 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:25.812660 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:25.812668 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:25.812684 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:25.812710 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:25.812781 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:25.812813 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:25.813028 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:25.813162 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 20:30:25.813389 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":485,"threads_started":1}
I20260806 20:30:25.815850 7867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:25.817265 7867 log.cc:824] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:25.819501 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:25.820796 7867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394: No bootstrap required, opened a new log
I20260806 20:30:25.827747 7870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:25.825011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37259 (local address 127.0.0.1:41777)
0806 20:30:25.825195 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:25.825202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:25.825494 (+ 292us) server_negotiation.cc:407] Connection header received
0806 20:30:25.825686 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:25.825705 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:25.825778 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:25.825884 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:25.826950 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:25.826955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:25.826960 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:25.826984 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:25.827041 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:25.827064 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:25.827177 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:25.827337 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 20:30:25.827396 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:30:25.832544 7867 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8d1f0682b44138afc179fa6f131394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } }
I20260806 20:30:25.833184 7867 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:25.833310 7867 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8d1f0682b44138afc179fa6f131394, State: Initialized, Role: FOLLOWER
I20260806 20:30:25.834044 7867 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8d1f0682b44138afc179fa6f131394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } }
I20260806 20:30:25.834373 7867 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:25.834528 7867 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:25.834730 7867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:25.838439 7867 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8d1f0682b44138afc179fa6f131394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } }
I20260806 20:30:25.839087 7867 leader_election.cc:302] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb8d1f0682b44138afc179fa6f131394; no voters:
I20260806 20:30:25.839862 7867 leader_election.cc:288] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:25.840008 7880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:25.840796 7880 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [term 1 LEADER]: Becoming Leader. State: Replica: fb8d1f0682b44138afc179fa6f131394, State: Running, Role: LEADER
I20260806 20:30:25.841421 7880 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8d1f0682b44138afc179fa6f131394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } }
I20260806 20:30:25.842549 7867 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:25.844956 7882 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb8d1f0682b44138afc179fa6f131394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8d1f0682b44138afc179fa6f131394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } } }
I20260806 20:30:25.845671 7882 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:25.852617 7883 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb8d1f0682b44138afc179fa6f131394. Latest consensus state: current_term: 1 leader_uuid: "fb8d1f0682b44138afc179fa6f131394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8d1f0682b44138afc179fa6f131394" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } } }
I20260806 20:30:25.853004 7883 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:25.853650 7890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:25.859030 7890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:25.871096 7890 catalog_manager.cc:1347] Generated new cluster ID: 506a8ea1fadf4629a92f06907f4dc2d4
I20260806 20:30:25.871227 7890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:25.891484 7890 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:25.892047 7890 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:25.902674 7890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb8d1f0682b44138afc179fa6f131394: Generated new TSK 0
I20260806 20:30:25.903548 7890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:25.913003 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7794
W20260806 20:30:25.914363 7862 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36187
W20260806 20:30:25.915104 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7794: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:25.915280 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 7794
2026-08-06T20:30:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2131 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:26 dist-test-slave-6k22 krb5kdc[7905](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:26 dist-test-slave-6k22 krb5kdc[7905](info): set up 2 sockets
Aug 06 20:30:26 dist-test-slave-6k22 krb5kdc[7905](info): commencing operation
krb5kdc: starting...
W20260806 20:30:27.092780 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:27 dist-test-slave-6k22 krb5kdc[7905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:27.428223 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:27.789003 7918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:27.789330 7918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:27.789443 7918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:27.797778 7918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:27.797895 7918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:27.797943 7918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:27.798014 7918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:27.798058 7918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:27.808139 7918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:27.809409 7918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:27.810726 7918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:27.822880 7926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:27.822888 7928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:27.823156 7925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:27.823941 7918 server_base.cc:1034] running on GCE node
I20260806 20:30:27.824741 7918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:27.826251 7918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:27.827497 7918 hybrid_clock.cc:648] HybridClock initialized: now 1786048227827446 us; error 46 us; skew 500 ppm
I20260806 20:30:27.827919 7918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:27 dist-test-slave-6k22 krb5kdc[7905](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048227, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:27.831238 7918 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:27.832716 7918 webserver.cc:460] Webserver started at http://127.0.0.1:42441/ using document root <none> and password file <none>
I20260806 20:30:27.833365 7918 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:27.833474 7918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:27.833719 7918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:27.836762 7918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "5ba4c87b994e442b8ca39df768f21eef"
format_stamp: "Formatted at 2026-08-06 20:30:27 on dist-test-slave-6k22"
server_key: "3662587085299c697d8f07249d5965c6"
server_key_iv: "b394541c98f02e7f2ba6c27f48b0e2e3"
server_key_version: "encryptionkey@0"
I20260806 20:30:27.837519 7918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "5ba4c87b994e442b8ca39df768f21eef"
format_stamp: "Formatted at 2026-08-06 20:30:27 on dist-test-slave-6k22"
server_key: "3662587085299c697d8f07249d5965c6"
server_key_iv: "b394541c98f02e7f2ba6c27f48b0e2e3"
server_key_version: "encryptionkey@0"
I20260806 20:30:27.842851 7918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260806 20:30:27.846481 7935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:27.847978 7918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:30:27.848127 7918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "5ba4c87b994e442b8ca39df768f21eef"
format_stamp: "Formatted at 2026-08-06 20:30:27 on dist-test-slave-6k22"
server_key: "3662587085299c697d8f07249d5965c6"
server_key_iv: "b394541c98f02e7f2ba6c27f48b0e2e3"
server_key_version: "encryptionkey@0"
I20260806 20:30:27.848316 7918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:27.860000 7918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:27.864353 7918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:27.864642 7918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:27.883279 7918 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42111
I20260806 20:30:27.883309 7993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42111 every 8 connection(s)
I20260806 20:30:27.884781 7918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:27.888648 7994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:27.890937 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 7918
I20260806 20:30:27.891131 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:27.891923 17847 external_mini_cluster.cc:1428] Setting key 1c48725aaf03b64357a52d0eb7734fec
I20260806 20:30:27.902616 7994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef: Bootstrap starting.
I20260806 20:30:27.905874 7994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef: Neither blocks nor log segments found. Creating new log.
Aug 06 20:30:27 dist-test-slave-6k22 krb5kdc[7905](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 20:30:27.906973 7994 log.cc:824] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:27.910975 7994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef: No bootstrap required, opened a new log
I20260806 20:30:27.916353 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:27.894023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:46051 (local address 127.0.0.1:42111)
0806 20:30:27.894903 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:27.894916 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:27.894958 (+ 42us) server_negotiation.cc:407] Connection header received
0806 20:30:27.896600 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:27.896624 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:27.897045 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:27.897449 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:27.899061 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:27.900522 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:27.901812 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:27.902505 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:27.902711 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:27.907078 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:27.907093 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:27.907113 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:27.907160 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:27.911427 (+ 4267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:27.912333 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:27.912343 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:27.912351 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:27.912573 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:27.913034 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:27.913039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:27.913043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:27.913768 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:27.914105 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:27.914549 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 20:30:27.914904 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":404,"threads_started":1}
I20260806 20:30:27.920499 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:27.921216 7994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba4c87b994e442b8ca39df768f21eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260806 20:30:27.921586 7994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:27.921691 7994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ba4c87b994e442b8ca39df768f21eef, State: Initialized, Role: FOLLOWER
I20260806 20:30:27.922292 7994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba4c87b994e442b8ca39df768f21eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260806 20:30:27.922542 7994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:27.922688 7994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:27.922865 7994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:27.925879 7994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba4c87b994e442b8ca39df768f21eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260806 20:30:27.926386 7994 leader_election.cc:302] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ba4c87b994e442b8ca39df768f21eef; no voters:
I20260806 20:30:27.927037 7994 leader_election.cc:288] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:27.927145 8005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:27.929713 7994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:27.933914 8005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [term 1 LEADER]: Becoming Leader. State: Replica: 5ba4c87b994e442b8ca39df768f21eef, State: Running, Role: LEADER
I20260806 20:30:27.934716 8005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba4c87b994e442b8ca39df768f21eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } }
I20260806 20:30:27.938539 8007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ba4c87b994e442b8ca39df768f21eef. Latest consensus state: current_term: 1 leader_uuid: "5ba4c87b994e442b8ca39df768f21eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba4c87b994e442b8ca39df768f21eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } } }
I20260806 20:30:27.939007 8007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:27.939663 8006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ba4c87b994e442b8ca39df768f21eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ba4c87b994e442b8ca39df768f21eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42111 } } }
I20260806 20:30:27.940021 8006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:27.940678 8014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:27.942705 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:27.925165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:55217 (local address 127.0.0.1:42111)
0806 20:30:27.925360 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:27.925369 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:27.925781 (+ 412us) server_negotiation.cc:407] Connection header received
0806 20:30:27.925979 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:27.925987 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:27.926076 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:27.926168 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:27.928123 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:27.928909 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:27.930290 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:27.930756 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:27.930895 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:27.933706 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:27.933711 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:27.933714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:27.933757 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:27.937129 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:27.938460 (+ 1331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:27.938467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:27.938470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:27.938647 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:27.939227 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:27.939233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:27.939237 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:27.939802 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:27.940182 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:27.941889 (+ 1707us) server_negotiation.cc:299] Negotiation successful
0806 20:30:27.942145 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 20:30:27.952248 8014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:27.961133 8014 catalog_manager.cc:1347] Generated new cluster ID: 7bfe446105684b7b965abc82c5f2658f
I20260806 20:30:27.961257 8014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:27.970211 8014 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:27.971509 8014 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:27.977355 8014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ba4c87b994e442b8ca39df768f21eef: Generated new TSK 0
I20260806 20:30:27.978139 8014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:27.991364 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 7918
I20260806 20:30:27.992758 7988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:28.084391 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 7918
W20260806 20:30:28.106568 7903 connection.cc:535] client connection to 127.0.0.1:42111 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42111): Connection reset by peer (error 104)
2026-08-06T20:30:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2170 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:28 dist-test-slave-6k22 krb5kdc[8027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:28 dist-test-slave-6k22 krb5kdc[8027](info): set up 2 sockets
Aug 06 20:30:28 dist-test-slave-6k22 krb5kdc[8027](info): commencing operation
krb5kdc: starting...
W20260806 20:30:29.268210 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.005s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:29 dist-test-slave-6k22 krb5kdc[8027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:29.607316 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:29.954380 8043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:29.954671 8043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:29.954855 8043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:29.962158 8043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:29.962276 8043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:29.962324 8043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:29.962425 8043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:29.962491 8043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:29.972579 8043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:29.973937 8043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:29.975306 8043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:29.987449 8050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:29.987378 8049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:29.987381 8052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:29.988565 8043 server_base.cc:1034] running on GCE node
I20260806 20:30:29.989363 8043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:29.990821 8043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:29.992057 8043 hybrid_clock.cc:648] HybridClock initialized: now 1786048229992016 us; error 56 us; skew 500 ppm
I20260806 20:30:29.992436 8043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:29 dist-test-slave-6k22 krb5kdc[8027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048229, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:30.001955 8043 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:30.003623 8043 webserver.cc:460] Webserver started at http://127.0.0.1:46537/ using document root <none> and password file <none>
I20260806 20:30:30.004362 8043 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:30.004522 8043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:30.004863 8043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:30.008131 8043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "d909b29a042f4440b40b34b52f88aa11"
format_stamp: "Formatted at 2026-08-06 20:30:30 on dist-test-slave-6k22"
server_key: "ef784b5086584bee58d67997282b6120"
server_key_iv: "1354cb00c9a0ff7de8ae2709fa8ec511"
server_key_version: "encryptionkey@0"
I20260806 20:30:30.008955 8043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "d909b29a042f4440b40b34b52f88aa11"
format_stamp: "Formatted at 2026-08-06 20:30:30 on dist-test-slave-6k22"
server_key: "ef784b5086584bee58d67997282b6120"
server_key_iv: "1354cb00c9a0ff7de8ae2709fa8ec511"
server_key_version: "encryptionkey@0"
I20260806 20:30:30.014429 8043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260806 20:30:30.018003 8060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:30.019441 8043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:30:30.019659 8043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "d909b29a042f4440b40b34b52f88aa11"
format_stamp: "Formatted at 2026-08-06 20:30:30 on dist-test-slave-6k22"
server_key: "ef784b5086584bee58d67997282b6120"
server_key_iv: "1354cb00c9a0ff7de8ae2709fa8ec511"
server_key_version: "encryptionkey@0"
I20260806 20:30:30.019877 8043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:30.031836 8043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:30.036312 8043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:30.036597 8043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:30.055622 8043 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36103
I20260806 20:30:30.055667 8117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36103 every 8 connection(s)
I20260806 20:30:30.057085 8043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:30.059780 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 8043
I20260806 20:30:30.059993 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:30.060542 17847 external_mini_cluster.cc:1428] Setting key c552617aac7261c472fc53bd02014b0a
I20260806 20:30:30.061348 8118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:30.074142 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11: Bootstrap starting.
I20260806 20:30:30.075083 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:30.062481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:45023 (local address 127.0.0.1:36103)
0806 20:30:30.063214 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:30.063226 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:30.063268 (+ 42us) server_negotiation.cc:407] Connection header received
0806 20:30:30.064934 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:30.064956 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:30.065367 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:30.065793 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:30.066346 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:30.067958 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:30.069609 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:30.070388 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:30.070639 (+ 251us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:30.072260 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:30.072270 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:30.072290 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:30.072933 (+ 643us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:30.073022 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:30.073052 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:30.073277 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:30.073621 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 20:30:30.073816 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":316,"threads_started":1}
I20260806 20:30:30.077976 8118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:30.079268 8118 log.cc:824] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:30.079335 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:30.082129 8118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11: No bootstrap required, opened a new log
I20260806 20:30:30.090816 8121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:30.083858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:59503 (local address 127.0.0.1:36103)
0806 20:30:30.084048 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:30.084054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:30.084405 (+ 351us) server_negotiation.cc:407] Connection header received
0806 20:30:30.084553 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:30.084564 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:30.084659 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:30.084793 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:30.085300 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:30.086491 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:30.087906 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:30.088640 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:30.088787 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:30.089948 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:30.089956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:30.089963 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:30.089997 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:30.090050 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:30.090076 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:30.090193 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:30.090363 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 20:30:30.090438 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:30:30.092156 8118 raft_consensus.cc:348] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d909b29a042f4440b40b34b52f88aa11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36103 } }
I20260806 20:30:30.092569 8118 raft_consensus.cc:374] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:30.092676 8118 raft_consensus.cc:729] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d909b29a042f4440b40b34b52f88aa11, State: Initialized, Role: FOLLOWER
I20260806 20:30:30.093361 8118 consensus_queue.cc:260] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d909b29a042f4440b40b34b52f88aa11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36103 } }
I20260806 20:30:30.093604 8118 raft_consensus.cc:388] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:30.093750 8118 raft_consensus.cc:482] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:30.093927 8118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:30.096282 8118 raft_consensus.cc:504] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d909b29a042f4440b40b34b52f88aa11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36103 } }
I20260806 20:30:30.096805 8118 leader_election.cc:302] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d909b29a042f4440b40b34b52f88aa11; no voters:
I20260806 20:30:30.097409 8118 leader_election.cc:288] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:30.097559 8128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:30.098361 8128 raft_consensus.cc:686] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [term 1 LEADER]: Becoming Leader. State: Replica: d909b29a042f4440b40b34b52f88aa11, State: Running, Role: LEADER
I20260806 20:30:30.098974 8128 consensus_queue.cc:237] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d909b29a042f4440b40b34b52f88aa11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36103 } }
I20260806 20:30:30.099697 8118 sys_catalog.cc:564] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:30.101974 8129 sys_catalog.cc:455] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d909b29a042f4440b40b34b52f88aa11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d909b29a042f4440b40b34b52f88aa11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36103 } } }
I20260806 20:30:30.102272 8130 sys_catalog.cc:455] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d909b29a042f4440b40b34b52f88aa11. Latest consensus state: current_term: 1 leader_uuid: "d909b29a042f4440b40b34b52f88aa11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d909b29a042f4440b40b34b52f88aa11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36103 } } }
I20260806 20:30:30.102514 8129 sys_catalog.cc:458] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:30.102599 8130 sys_catalog.cc:458] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:30.104954 8135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:30.110399 8135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:30.119868 8135 catalog_manager.cc:1347] Generated new cluster ID: f275bbbbd1a948298e89832a1b01ae18
I20260806 20:30:30.119992 8135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:30.135396 8135 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:30.137280 8135 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:30.152269 8135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d909b29a042f4440b40b34b52f88aa11: Generated new TSK 0
I20260806 20:30:30.153340 8135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:30.164414 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8043
W20260806 20:30:30.166355 8113 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.223:45023
W20260806 20:30:30.167194 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8043: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:30.167373 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 8043
W20260806 20:30:30.190469 8025 connection.cc:535] client connection to 127.0.0.1:36103 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36103): Connection reset by peer (error 104)
2026-08-06T20:30:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2083 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:30 dist-test-slave-6k22 krb5kdc[8149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:30 dist-test-slave-6k22 krb5kdc[8149](info): set up 2 sockets
Aug 06 20:30:30 dist-test-slave-6k22 krb5kdc[8149](info): commencing operation
krb5kdc: starting...
W20260806 20:30:31.348990 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.007s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:31 dist-test-slave-6k22 krb5kdc[8149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:31.682510 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:32.034842 8163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:32.035130 8163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:32.035341 8163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:32.042641 8163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:32.042755 8163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:32.042837 8163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:32.042945 8163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:32.043031 8163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:32.053202 8163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:32.054540 8163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:32.055941 8163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:32.067999 8172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:32.067965 8170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:32.068253 8168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:32.068799 8163 server_base.cc:1034] running on GCE node
I20260806 20:30:32.069638 8163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:32.071120 8163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:32.072353 8163 hybrid_clock.cc:648] HybridClock initialized: now 1786048232072305 us; error 48 us; skew 500 ppm
I20260806 20:30:32.072691 8163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:32 dist-test-slave-6k22 krb5kdc[8149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048232, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:32.076000 8163 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:32.077479 8163 webserver.cc:460] Webserver started at http://127.0.0.1:34149/ using document root <none> and password file <none>
I20260806 20:30:32.078121 8163 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:32.078227 8163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:32.078501 8163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:32.081537 8163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "952c943bfce9467a91b56bd34fa8560e"
format_stamp: "Formatted at 2026-08-06 20:30:32 on dist-test-slave-6k22"
server_key: "1709b4d0e4718bf7318de8f2a4bc2767"
server_key_iv: "8979e6d01fca51064d2447179335418e"
server_key_version: "encryptionkey@0"
I20260806 20:30:32.082271 8163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "952c943bfce9467a91b56bd34fa8560e"
format_stamp: "Formatted at 2026-08-06 20:30:32 on dist-test-slave-6k22"
server_key: "1709b4d0e4718bf7318de8f2a4bc2767"
server_key_iv: "8979e6d01fca51064d2447179335418e"
server_key_version: "encryptionkey@0"
I20260806 20:30:32.087494 8163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260806 20:30:32.091073 8180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:32.092789 8163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:30:32.092954 8163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "952c943bfce9467a91b56bd34fa8560e"
format_stamp: "Formatted at 2026-08-06 20:30:32 on dist-test-slave-6k22"
server_key: "1709b4d0e4718bf7318de8f2a4bc2767"
server_key_iv: "8979e6d01fca51064d2447179335418e"
server_key_version: "encryptionkey@0"
I20260806 20:30:32.093140 8163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:32.098989 8163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:32.103000 8163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:32.103253 8163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:32.122671 8163 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36429
I20260806 20:30:32.122768 8236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36429 every 8 connection(s)
I20260806 20:30:32.124086 8163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:32.124698 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 8163
I20260806 20:30:32.124898 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:32.125515 17847 external_mini_cluster.cc:1428] Setting key 3d239eface5ba1dd1ba7c2d88e960d4d
I20260806 20:30:32.128494 8238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:32.133366 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:32.127481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:49165 (local address 127.0.0.1:36429)
0806 20:30:32.128319 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:32.128330 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:32.128370 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:30:32.130034 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:32.130056 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:32.130377 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:32.130698 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:32.131802 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:32.131808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:32.131820 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:32.131833 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:32.131905 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:32.131925 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:32.132094 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:32.132233 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 20:30:32.132447 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":347,"threads_started":1}
I20260806 20:30:32.137260 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:32.140503 8238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e: Bootstrap starting.
I20260806 20:30:32.144415 8238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:32.144586 8242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:32.141722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:41443 (local address 127.0.0.1:36429)
0806 20:30:32.141941 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:32.141950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:32.142426 (+ 476us) server_negotiation.cc:407] Connection header received
0806 20:30:32.142593 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:32.142597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:32.142647 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:32.142724 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:32.143724 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:32.143728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:32.143754 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:32.143769 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:32.143807 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:32.143821 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:32.143915 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:32.144076 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 20:30:32.144123 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 20:30:32.145807 8238 log.cc:824] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:32.148787 8238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e: No bootstrap required, opened a new log
I20260806 20:30:32.156260 8238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952c943bfce9467a91b56bd34fa8560e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36429 } }
I20260806 20:30:32.156602 8238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:32.156675 8238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 952c943bfce9467a91b56bd34fa8560e, State: Initialized, Role: FOLLOWER
I20260806 20:30:32.157218 8238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952c943bfce9467a91b56bd34fa8560e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36429 } }
I20260806 20:30:32.157425 8238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:32.157521 8238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:32.157645 8238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:32.159780 8238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952c943bfce9467a91b56bd34fa8560e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36429 } }
I20260806 20:30:32.160203 8238 leader_election.cc:302] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 952c943bfce9467a91b56bd34fa8560e; no voters:
I20260806 20:30:32.160741 8238 leader_election.cc:288] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:32.160977 8250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:32.161820 8250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [term 1 LEADER]: Becoming Leader. State: Replica: 952c943bfce9467a91b56bd34fa8560e, State: Running, Role: LEADER
I20260806 20:30:32.162382 8250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952c943bfce9467a91b56bd34fa8560e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36429 } }
I20260806 20:30:32.162942 8238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:32.165349 8251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "952c943bfce9467a91b56bd34fa8560e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952c943bfce9467a91b56bd34fa8560e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36429 } } }
I20260806 20:30:32.165845 8251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:32.165386 8252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 952c943bfce9467a91b56bd34fa8560e. Latest consensus state: current_term: 1 leader_uuid: "952c943bfce9467a91b56bd34fa8560e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "952c943bfce9467a91b56bd34fa8560e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36429 } } }
I20260806 20:30:32.166374 8252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:32.168184 8256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:32.180302 8256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:32.192153 8256 catalog_manager.cc:1347] Generated new cluster ID: 155fffa0583f418e8d05e41088148959
I20260806 20:30:32.192291 8256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:32.206115 8256 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:32.206562 8256 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:32.214792 8256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 952c943bfce9467a91b56bd34fa8560e: Generated new TSK 0
I20260806 20:30:32.215601 8256 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:32.220942 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8163
W20260806 20:30:32.222020 8232 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.223:49165
W20260806 20:30:32.222704 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8163: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:32.222853 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 8163
2026-08-06T20:30:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2052 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:32 dist-test-slave-6k22 krb5kdc[8273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:32 dist-test-slave-6k22 krb5kdc[8273](info): set up 2 sockets
Aug 06 20:30:32 dist-test-slave-6k22 krb5kdc[8273](info): commencing operation
krb5kdc: starting...
W20260806 20:30:33.429203 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:33 dist-test-slave-6k22 krb5kdc[8273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:33.774763 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38527
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:34.125300 8287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:34.125663 8287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:34.125850 8287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:34.133183 8287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:34.133350 8287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:34.133479 8287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:34.133587 8287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:34.133675 8287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:34.143640 8287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38527
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:34.145056 8287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:34.146452 8287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:34.158699 8292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:34.158639 8295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:34.158603 8293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:34.159613 8287 server_base.cc:1034] running on GCE node
I20260806 20:30:34.160460 8287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:34.161897 8287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:34.163105 8287 hybrid_clock.cc:648] HybridClock initialized: now 1786048234163067 us; error 27 us; skew 500 ppm
I20260806 20:30:34.163471 8287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:34 dist-test-slave-6k22 krb5kdc[8273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048234, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:34.166677 8287 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:34.168175 8287 webserver.cc:460] Webserver started at http://127.0.0.1:40993/ using document root <none> and password file <none>
I20260806 20:30:34.168808 8287 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:34.168921 8287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:34.169167 8287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:34.172132 8287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3"
format_stamp: "Formatted at 2026-08-06 20:30:34 on dist-test-slave-6k22"
server_key: "0533de2056e2d31cb801255d79f4a76f"
server_key_iv: "46d68c745eda11f31d26da945403329a"
server_key_version: "encryptionkey@0"
I20260806 20:30:34.173113 8287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3"
format_stamp: "Formatted at 2026-08-06 20:30:34 on dist-test-slave-6k22"
server_key: "0533de2056e2d31cb801255d79f4a76f"
server_key_iv: "46d68c745eda11f31d26da945403329a"
server_key_version: "encryptionkey@0"
I20260806 20:30:34.180336 8287 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260806 20:30:34.183913 8303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:34.185209 8287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:30:34.185401 8287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3"
format_stamp: "Formatted at 2026-08-06 20:30:34 on dist-test-slave-6k22"
server_key: "0533de2056e2d31cb801255d79f4a76f"
server_key_iv: "46d68c745eda11f31d26da945403329a"
server_key_version: "encryptionkey@0"
I20260806 20:30:34.185602 8287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:34.198593 8287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:34.202476 8287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:34.202759 8287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:34.221733 8287 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38527
I20260806 20:30:34.221765 8360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38527 every 8 connection(s)
I20260806 20:30:34.223280 8287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:34.226980 8361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:34.227988 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 8287
I20260806 20:30:34.228190 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:34.228667 17847 external_mini_cluster.cc:1428] Setting key 2f19f40a7cc8f936922b0f7753de8d45
I20260806 20:30:34.242871 8361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3: Bootstrap starting.
Aug 06 20:30:34 dist-test-slave-6k22 krb5kdc[8273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 20:30:34.246641 8361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:34.248153 8361 log.cc:824] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:34.251765 8361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3: No bootstrap required, opened a new log
I20260806 20:30:34.255829 8364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:34.230510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:46201 (local address 127.0.0.1:38527)
0806 20:30:34.231326 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:34.231338 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:34.231381 (+ 43us) server_negotiation.cc:407] Connection header received
0806 20:30:34.233104 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:34.233127 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:34.233539 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:34.233890 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:34.236143 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:34.237450 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:34.239718 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:34.240425 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:34.240558 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:34.246450 (+ 5892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:34.246464 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:34.246486 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:34.246544 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:34.250620 (+ 4076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:34.251551 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:34.251587 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:34.251596 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:34.251848 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:34.252357 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:34.252362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:34.252365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:34.253090 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:34.253425 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:34.253982 (+ 557us) server_negotiation.cc:299] Negotiation successful
0806 20:30:34.254357 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":385,"threads_started":1}
I20260806 20:30:34.260273 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:34.261660 8361 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260806 20:30:34.262033 8361 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:34.262132 8361 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca90ca35ef3841e3ab1d3a7a8a255ba3, State: Initialized, Role: FOLLOWER
I20260806 20:30:34.262776 8361 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260806 20:30:34.263025 8361 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:34.263197 8361 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:34.263381 8361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:34.266307 8361 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260806 20:30:34.266816 8361 leader_election.cc:302] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca90ca35ef3841e3ab1d3a7a8a255ba3; no voters:
I20260806 20:30:34.267524 8361 leader_election.cc:288] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:34.267741 8371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:34.270332 8361 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:34.274942 8371 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [term 1 LEADER]: Becoming Leader. State: Replica: ca90ca35ef3841e3ab1d3a7a8a255ba3, State: Running, Role: LEADER
I20260806 20:30:34.276055 8371 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } }
I20260806 20:30:34.279420 8372 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } } }
I20260806 20:30:34.279858 8372 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:34.284211 8385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:34.284622 8364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:34.264894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:33767 (local address 127.0.0.1:38527)
0806 20:30:34.265070 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:34.265078 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:34.265537 (+ 459us) server_negotiation.cc:407] Connection header received
0806 20:30:34.265764 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:34.265769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:34.265840 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:34.265946 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:34.267582 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:34.269049 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:34.270446 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:34.271057 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:34.271191 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:34.274134 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:34.274146 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:34.274152 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:34.274200 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:34.279000 (+ 4800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:34.280463 (+ 1463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:34.280471 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:34.280478 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:34.280705 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:34.281468 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:34.281476 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:34.281482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:34.282058 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:34.282426 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:34.283916 (+ 1490us) server_negotiation.cc:299] Negotiation successful
0806 20:30:34.284182 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 20:30:34.284583 8373 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca90ca35ef3841e3ab1d3a7a8a255ba3. Latest consensus state: current_term: 1 leader_uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca90ca35ef3841e3ab1d3a7a8a255ba3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38527 } } }
I20260806 20:30:34.285341 8373 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:34.289968 8385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:34.299540 8385 catalog_manager.cc:1347] Generated new cluster ID: ec28e061b9e7434a9bed6024ad8c7e3b
I20260806 20:30:34.299739 8385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:34.330878 8385 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:34.332850 8385 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:34.340875 8385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca90ca35ef3841e3ab1d3a7a8a255ba3: Generated new TSK 0
I20260806 20:30:34.341696 8385 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:34.356422 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8287
I20260806 20:30:34.357654 8356 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:34.452116 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 8287
W20260806 20:30:34.473484 8270 connection.cc:535] client connection to 127.0.0.1:38527 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38527): Connection reset by peer (error 104)
2026-08-06T20:30:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2234 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:34 dist-test-slave-6k22 krb5kdc[8397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:34 dist-test-slave-6k22 krb5kdc[8397](info): set up 2 sockets
Aug 06 20:30:34 dist-test-slave-6k22 krb5kdc[8397](info): commencing operation
krb5kdc: starting...
W20260806 20:30:35.728489 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:35 dist-test-slave-6k22 krb5kdc[8397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:36.079196 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:36.445998 8412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:36.446331 8412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:36.446440 8412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:36.453987 8412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:36.454115 8412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:36.454177 8412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:36.454250 8412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:36.454303 8412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:36.464423 8412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41141
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:36.465741 8412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:36.467072 8412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:36.480633 8420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:36.480553 8418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:36.481231 8417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:36.481535 8412 server_base.cc:1034] running on GCE node
I20260806 20:30:36.482434 8412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:36.484210 8412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:36.485504 8412 hybrid_clock.cc:648] HybridClock initialized: now 1786048236485425 us; error 63 us; skew 500 ppm
I20260806 20:30:36.485985 8412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:36 dist-test-slave-6k22 krb5kdc[8397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048236, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:36.489878 8412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:36.491746 8412 webserver.cc:460] Webserver started at http://127.0.0.1:36059/ using document root <none> and password file <none>
I20260806 20:30:36.492540 8412 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:36.492735 8412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:36.493096 8412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:36.496477 8412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "dc36b64049d1437593aeab867f55d240"
format_stamp: "Formatted at 2026-08-06 20:30:36 on dist-test-slave-6k22"
server_key: "90f9c01d64d9cb4b38db334c1cc55cbc"
server_key_iv: "490ddd445623e3653862eb357336f4c7"
server_key_version: "encryptionkey@0"
I20260806 20:30:36.497326 8412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "dc36b64049d1437593aeab867f55d240"
format_stamp: "Formatted at 2026-08-06 20:30:36 on dist-test-slave-6k22"
server_key: "90f9c01d64d9cb4b38db334c1cc55cbc"
server_key_iv: "490ddd445623e3653862eb357336f4c7"
server_key_version: "encryptionkey@0"
I20260806 20:30:36.503427 8412 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260806 20:30:36.507673 8427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:36.509321 8412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:30:36.509567 8412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "dc36b64049d1437593aeab867f55d240"
format_stamp: "Formatted at 2026-08-06 20:30:36 on dist-test-slave-6k22"
server_key: "90f9c01d64d9cb4b38db334c1cc55cbc"
server_key_iv: "490ddd445623e3653862eb357336f4c7"
server_key_version: "encryptionkey@0"
I20260806 20:30:36.509845 8412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:36.527842 8412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:36.533169 8412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:36.533519 8412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:36.554921 8412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41141
I20260806 20:30:36.554955 8482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41141 every 8 connection(s)
I20260806 20:30:36.556607 8412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:36.560529 8483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:36.564563 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 8412
I20260806 20:30:36.564769 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:36.565359 17847 external_mini_cluster.cc:1428] Setting key bad3ea374ef3e16112f1196636ef7696
I20260806 20:30:36.573570 8483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240: Bootstrap starting.
I20260806 20:30:36.576784 8483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:36.577893 8483 log.cc:824] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:36.579895 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:36.567268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:35373 (local address 127.0.0.1:41141)
0806 20:30:36.568139 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:36.568151 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:36.568197 (+ 46us) server_negotiation.cc:407] Connection header received
0806 20:30:36.570051 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:36.570074 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:36.570551 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:36.571001 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:36.571708 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:36.573175 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:36.574752 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:36.575532 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:36.575818 (+ 286us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:36.577248 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:36.577257 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:36.577273 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:36.577856 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:36.577938 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:36.577964 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:36.578159 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:36.578518 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 20:30:36.578724 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":412,"threads_started":1}
I20260806 20:30:36.580897 8483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240: No bootstrap required, opened a new log
I20260806 20:30:36.584092 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:36.590936 8483 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc36b64049d1437593aeab867f55d240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } }
I20260806 20:30:36.591437 8483 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:36.591545 8483 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc36b64049d1437593aeab867f55d240, State: Initialized, Role: FOLLOWER
I20260806 20:30:36.592206 8483 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc36b64049d1437593aeab867f55d240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } }
I20260806 20:30:36.592463 8483 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:36.592624 8483 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:36.592829 8483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:36.596040 8483 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc36b64049d1437593aeab867f55d240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } }
I20260806 20:30:36.596714 8483 leader_election.cc:302] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc36b64049d1437593aeab867f55d240; no voters:
I20260806 20:30:36.597273 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:36.589611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:45819 (local address 127.0.0.1:41141)
0806 20:30:36.589863 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:36.589870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:36.590225 (+ 355us) server_negotiation.cc:407] Connection header received
0806 20:30:36.590388 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:36.590393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:36.590458 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:36.590552 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:36.591218 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:36.592417 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:36.594383 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:36.594909 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:36.595010 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:36.596355 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:36.596360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:36.596364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:36.596383 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:36.596441 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:36.596458 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:36.596549 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:36.596786 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 20:30:36.596834 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 20:30:36.597481 8483 leader_election.cc:288] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:36.597608 8495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:36.599290 8495 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [term 1 LEADER]: Becoming Leader. State: Replica: dc36b64049d1437593aeab867f55d240, State: Running, Role: LEADER
I20260806 20:30:36.600134 8483 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:36.600031 8495 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc36b64049d1437593aeab867f55d240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } }
I20260806 20:30:36.603194 8497 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc36b64049d1437593aeab867f55d240. Latest consensus state: current_term: 1 leader_uuid: "dc36b64049d1437593aeab867f55d240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc36b64049d1437593aeab867f55d240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } } }
I20260806 20:30:36.603322 8496 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc36b64049d1437593aeab867f55d240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc36b64049d1437593aeab867f55d240" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41141 } } }
I20260806 20:30:36.603643 8497 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:36.603729 8496 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:36.604802 8505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:36.610425 8505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:36.619170 8505 catalog_manager.cc:1347] Generated new cluster ID: a55c2739289a4135ab94c156e38ea992
I20260806 20:30:36.619290 8505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:36.647502 8505 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:36.649600 8505 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:36.668207 8505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc36b64049d1437593aeab867f55d240: Generated new TSK 0
I20260806 20:30:36.669431 8505 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:36.682022 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8412
W20260806 20:30:36.684017 8478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.223:35373
W20260806 20:30:36.684876 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8412: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:36.685057 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 8412
W20260806 20:30:36.706634 8394 connection.cc:535] client connection to 127.0.0.1:41141 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41141): Connection reset by peer (error 104)
2026-08-06T20:30:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2228 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:36 dist-test-slave-6k22 krb5kdc[8518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:36 dist-test-slave-6k22 krb5kdc[8518](info): set up 2 sockets
Aug 06 20:30:36 dist-test-slave-6k22 krb5kdc[8518](info): commencing operation
krb5kdc: starting...
W20260806 20:30:37.896730 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:38 dist-test-slave-6k22 krb5kdc[8518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 20:30:38.238961 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:38.592262 8532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:38.592592 8532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:38.592689 8532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:38.600085 8532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:38.600210 8532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:38.600256 8532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:38.600334 8532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:38.600380 8532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:38.610430 8532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:38.611860 8532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:38.613142 8532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:38.625512 8540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:38.625605 8539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:38.625582 8542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:38.626493 8532 server_base.cc:1034] running on GCE node
I20260806 20:30:38.627254 8532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:38.628685 8532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:38.629894 8532 hybrid_clock.cc:648] HybridClock initialized: now 1786048238629836 us; error 54 us; skew 500 ppm
I20260806 20:30:38.630254 8532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:38 dist-test-slave-6k22 krb5kdc[8518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:38.633533 8532 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 20:30:38.635063 8532 webserver.cc:460] Webserver started at http://127.0.0.1:35079/ using document root <none> and password file <none>
I20260806 20:30:38.635835 8532 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:38.635972 8532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:38.636292 8532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:38.639322 8532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "d9b29faca1ad48499b0994ebf206ad96"
format_stamp: "Formatted at 2026-08-06 20:30:38 on dist-test-slave-6k22"
server_key: "47e8f5692d70989c97fffc91a8ae0549"
server_key_iv: "3ff6f804113a7197261f98ab365c785a"
server_key_version: "encryptionkey@0"
I20260806 20:30:38.640142 8532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "d9b29faca1ad48499b0994ebf206ad96"
format_stamp: "Formatted at 2026-08-06 20:30:38 on dist-test-slave-6k22"
server_key: "47e8f5692d70989c97fffc91a8ae0549"
server_key_iv: "3ff6f804113a7197261f98ab365c785a"
server_key_version: "encryptionkey@0"
I20260806 20:30:38.645684 8532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 20:30:38.649132 8549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:38.650439 8532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:30:38.650629 8532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "d9b29faca1ad48499b0994ebf206ad96"
format_stamp: "Formatted at 2026-08-06 20:30:38 on dist-test-slave-6k22"
server_key: "47e8f5692d70989c97fffc91a8ae0549"
server_key_iv: "3ff6f804113a7197261f98ab365c785a"
server_key_version: "encryptionkey@0"
I20260806 20:30:38.650826 8532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:38.656591 8532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:38.660982 8532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:38.661279 8532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:38.680249 8532 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46807
I20260806 20:30:38.680305 8603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46807 every 8 connection(s)
I20260806 20:30:38.681757 8532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:38.682260 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 8532
I20260806 20:30:38.682459 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:38.683179 17847 external_mini_cluster.cc:1428] Setting key 6dc2df43075ab2b6bdd5d6bb82842f63
I20260806 20:30:38.686581 8604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:38.692521 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:38.685328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:59019 (local address 127.0.0.1:46807)
0806 20:30:38.686181 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:38.686200 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:38.686251 (+ 51us) server_negotiation.cc:407] Connection header received
0806 20:30:38.688588 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:38.688616 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:38.689029 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:38.689482 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:38.690515 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:38.690523 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:38.690539 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:38.690556 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:38.690646 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:38.690674 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:38.690891 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:38.691040 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 20:30:38.691231 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":357,"threads_started":1}
I20260806 20:30:38.696633 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:38.700052 8604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96: Bootstrap starting.
I20260806 20:30:38.703847 8604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:38.704030 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:38.701229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.223:43421 (local address 127.0.0.1:46807)
0806 20:30:38.701429 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:38.701436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:38.701812 (+ 376us) server_negotiation.cc:407] Connection header received
0806 20:30:38.702025 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:38.702030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:38.702090 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:38.702197 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:38.703263 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:38.703268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:38.703272 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:38.703296 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:38.703337 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:38.703356 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:38.703486 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:38.703635 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 20:30:38.703719 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 20:30:38.705300 8604 log.cc:824] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:38.708765 8604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96: No bootstrap required, opened a new log
I20260806 20:30:38.717146 8604 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b29faca1ad48499b0994ebf206ad96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46807 } }
I20260806 20:30:38.717551 8604 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:38.717724 8604 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b29faca1ad48499b0994ebf206ad96, State: Initialized, Role: FOLLOWER
I20260806 20:30:38.718452 8604 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b29faca1ad48499b0994ebf206ad96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46807 } }
I20260806 20:30:38.718713 8604 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:38.718887 8604 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:38.719094 8604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:38.722074 8604 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b29faca1ad48499b0994ebf206ad96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46807 } }
I20260806 20:30:38.722713 8604 leader_election.cc:302] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9b29faca1ad48499b0994ebf206ad96; no voters:
I20260806 20:30:38.723513 8604 leader_election.cc:288] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:38.723696 8615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:38.724556 8615 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [term 1 LEADER]: Becoming Leader. State: Replica: d9b29faca1ad48499b0994ebf206ad96, State: Running, Role: LEADER
I20260806 20:30:38.725168 8615 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b29faca1ad48499b0994ebf206ad96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46807 } }
I20260806 20:30:38.726132 8604 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:38.728272 8616 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9b29faca1ad48499b0994ebf206ad96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b29faca1ad48499b0994ebf206ad96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46807 } } }
I20260806 20:30:38.728408 8617 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b29faca1ad48499b0994ebf206ad96. Latest consensus state: current_term: 1 leader_uuid: "d9b29faca1ad48499b0994ebf206ad96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b29faca1ad48499b0994ebf206ad96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46807 } } }
I20260806 20:30:38.728757 8616 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:38.728760 8617 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:38.729864 8623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:38.736359 8623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:38.746377 8623 catalog_manager.cc:1347] Generated new cluster ID: 7b39e28fc8e245cea5430ff2115c0fff
I20260806 20:30:38.746526 8623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:38.766135 8623 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:38.766685 8623 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:38.783496 8623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9b29faca1ad48499b0994ebf206ad96: Generated new TSK 0
I20260806 20:30:38.784637 8623 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:38.796504 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8532
W20260806 20:30:38.797720 8599 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.223:59019
W20260806 20:30:38.798553 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8532: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:38.798800 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 8532
2026-08-06T20:30:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2113 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:38 dist-test-slave-6k22 krb5kdc[8636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:38 dist-test-slave-6k22 krb5kdc[8636](info): set up 2 sockets
Aug 06 20:30:38 dist-test-slave-6k22 krb5kdc[8636](info): commencing operation
krb5kdc: starting...
W20260806 20:30:40.076651 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.007s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:40 dist-test-slave-6k22 krb5kdc[8636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:40.413180 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:42539
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:42539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:40.762943 8652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:40.763231 8652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:40.763435 8652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:40.770795 8652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:40.770931 8652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:40.771032 8652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:40.771147 8652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:40.771227 8652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:40.781138 8652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:42539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:42539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:40.782442 8652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:40.783823 8652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:40.796334 8659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:40.796550 8661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:40.796795 8658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:40.797099 8652 server_base.cc:1034] running on GCE node
I20260806 20:30:40.797838 8652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:40.799202 8652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:40.800406 8652 hybrid_clock.cc:648] HybridClock initialized: now 1786048240800352 us; error 41 us; skew 500 ppm
I20260806 20:30:40.800694 8652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:40 dist-test-slave-6k22 krb5kdc[8636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048240, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:40.809481 8652 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:40.810905 8652 webserver.cc:460] Webserver started at http://127.17.109.254:41449/ using document root <none> and password file <none>
I20260806 20:30:40.811539 8652 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:40.811712 8652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:40.811982 8652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:40.814980 8652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "7d09b04ccd5740fda3ff46c7546fb59d"
format_stamp: "Formatted at 2026-08-06 20:30:40 on dist-test-slave-6k22"
server_key: "846172a17bff56cb9773be6524626f56"
server_key_iv: "9d46b76a51f5934836e54b60be9aff99"
server_key_version: "encryptionkey@0"
I20260806 20:30:40.815784 8652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "7d09b04ccd5740fda3ff46c7546fb59d"
format_stamp: "Formatted at 2026-08-06 20:30:40 on dist-test-slave-6k22"
server_key: "846172a17bff56cb9773be6524626f56"
server_key_iv: "9d46b76a51f5934836e54b60be9aff99"
server_key_version: "encryptionkey@0"
I20260806 20:30:40.820858 8652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:30:40.824358 8670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:40.826120 8652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:40.826264 8652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "7d09b04ccd5740fda3ff46c7546fb59d"
format_stamp: "Formatted at 2026-08-06 20:30:40 on dist-test-slave-6k22"
server_key: "846172a17bff56cb9773be6524626f56"
server_key_iv: "9d46b76a51f5934836e54b60be9aff99"
server_key_version: "encryptionkey@0"
I20260806 20:30:40.826433 8652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:40.856822 8652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:40.861014 8652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:40.861330 8652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:40.880226 8652 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:42539
I20260806 20:30:40.880265 8724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:42539 every 8 connection(s)
I20260806 20:30:40.881628 8652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:40.885504 8725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:40.886255 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 8652
I20260806 20:30:40.886471 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:40.886919 17847 external_mini_cluster.cc:1428] Setting key ae4b588b51d57ce1bd59944f0e48457c
I20260806 20:30:40.899442 8725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d: Bootstrap starting.
Aug 06 20:30:40 dist-test-slave-6k22 krb5kdc[8636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:40.903151 8725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:40.904455 8725 log.cc:824] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:40.907609 8725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d: No bootstrap required, opened a new log
I20260806 20:30:40.911051 8728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:40.888854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52071 (local address 127.17.109.254:42539)
0806 20:30:40.889692 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:40.889704 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:40.889752 (+ 48us) server_negotiation.cc:407] Connection header received
0806 20:30:40.891447 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:40.891473 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:40.891935 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:40.892264 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:40.894153 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:40.895407 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:40.897338 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:40.898008 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:40.898136 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:40.903169 (+ 5033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:40.903177 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:40.903191 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:40.903226 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:40.906797 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:40.907619 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:40.907627 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:40.907641 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:40.907862 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:40.908337 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:40.908346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:40.908349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:40.908856 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:40.909226 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:40.909706 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 20:30:40.910037 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":374,"threads_started":1}
I20260806 20:30:40.915155 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:40.916595 8725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d09b04ccd5740fda3ff46c7546fb59d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42539 } }
I20260806 20:30:40.916971 8725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:40.917079 8725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d09b04ccd5740fda3ff46c7546fb59d, State: Initialized, Role: FOLLOWER
I20260806 20:30:40.917646 8725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d09b04ccd5740fda3ff46c7546fb59d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42539 } }
I20260806 20:30:40.918031 8725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:40.918193 8725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:40.918417 8725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:40.921149 8725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d09b04ccd5740fda3ff46c7546fb59d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42539 } }
I20260806 20:30:40.921643 8725 leader_election.cc:302] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d09b04ccd5740fda3ff46c7546fb59d; no voters:
I20260806 20:30:40.922420 8725 leader_election.cc:288] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:40.922590 8735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:40.925172 8725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:40.929287 8735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [term 1 LEADER]: Becoming Leader. State: Replica: 7d09b04ccd5740fda3ff46c7546fb59d, State: Running, Role: LEADER
I20260806 20:30:40.930070 8735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d09b04ccd5740fda3ff46c7546fb59d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42539 } }
I20260806 20:30:40.934142 8736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d09b04ccd5740fda3ff46c7546fb59d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d09b04ccd5740fda3ff46c7546fb59d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42539 } } }
I20260806 20:30:40.934588 8736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:40.936407 8737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d09b04ccd5740fda3ff46c7546fb59d. Latest consensus state: current_term: 1 leader_uuid: "7d09b04ccd5740fda3ff46c7546fb59d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d09b04ccd5740fda3ff46c7546fb59d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 42539 } } }
I20260806 20:30:40.937279 8737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:40.937702 8728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:40.919666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54237 (local address 127.17.109.254:42539)
0806 20:30:40.919839 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:40.919843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:40.920327 (+ 484us) server_negotiation.cc:407] Connection header received
0806 20:30:40.920521 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:40.920527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:40.920599 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:40.920699 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:40.922205 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:40.923615 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:40.924915 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:40.925474 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:40.925606 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:40.928517 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:40.928522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:40.928525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:40.928554 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:40.932728 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:40.933845 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:40.933864 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:40.933869 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:40.934033 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:40.934683 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:40.934688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:40.934692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:40.935240 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:40.936758 (+ 1518us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:40.937008 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 20:30:40.937246 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 20:30:40.940387 8745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:40.945978 8745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:40.955350 8745 catalog_manager.cc:1347] Generated new cluster ID: d2542656e96d43a69ca731d515b9b1e7
I20260806 20:30:40.955511 8745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:40.967290 8745 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:40.969492 8745 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:40.980535 8745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d09b04ccd5740fda3ff46c7546fb59d: Generated new TSK 0
I20260806 20:30:40.981426 8745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:40.992482 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8652
I20260806 20:30:40.993800 8720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:41.088807 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 8652
W20260806 20:30:41.109388 8634 connection.cc:535] client connection to 127.17.109.254:42539 recv error: Network error: failed to read from TLS socket (remote: 127.17.109.254:42539): Connection reset by peer (error 104)
2026-08-06T20:30:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2288 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:41 dist-test-slave-6k22 krb5kdc[8761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:41 dist-test-slave-6k22 krb5kdc[8761](info): set up 2 sockets
Aug 06 20:30:41 dist-test-slave-6k22 krb5kdc[8761](info): commencing operation
krb5kdc: starting...
W20260806 20:30:42.267395 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:42 dist-test-slave-6k22 krb5kdc[8761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:42.598800 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:44885
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:44885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:42.940594 8774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:42.940912 8774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:42.941083 8774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:42.948490 8774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:42.948604 8774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:42.948716 8774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:42.948830 8774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:42.948909 8774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:42.959082 8774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:44885
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:44885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:42.960564 8774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:42.961977 8774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:42.974424 8782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:42.974455 8783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:42.974532 8785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:42.975214 8774 server_base.cc:1034] running on GCE node
I20260806 20:30:42.976145 8774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:42.977739 8774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:42.978955 8774 hybrid_clock.cc:648] HybridClock initialized: now 1786048242978915 us; error 41 us; skew 500 ppm
I20260806 20:30:42.979324 8774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:42 dist-test-slave-6k22 krb5kdc[8761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048242, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:42.982555 8774 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:42.984223 8774 webserver.cc:460] Webserver started at http://127.17.109.254:39391/ using document root <none> and password file <none>
I20260806 20:30:42.984942 8774 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:42.985095 8774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:42.985405 8774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:42.988556 8774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "cab883e8e6314a419602231e30f887a4"
format_stamp: "Formatted at 2026-08-06 20:30:42 on dist-test-slave-6k22"
server_key: "ee3088cb39c2595b248bebbb399a522f"
server_key_iv: "8cb1afe58101bf806d35eaf4af5efc3a"
server_key_version: "encryptionkey@0"
I20260806 20:30:42.989351 8774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "cab883e8e6314a419602231e30f887a4"
format_stamp: "Formatted at 2026-08-06 20:30:42 on dist-test-slave-6k22"
server_key: "ee3088cb39c2595b248bebbb399a522f"
server_key_iv: "8cb1afe58101bf806d35eaf4af5efc3a"
server_key_version: "encryptionkey@0"
I20260806 20:30:42.994665 8774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:30:42.998379 8793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:42.999920 8774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 20:30:43.000129 8774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "cab883e8e6314a419602231e30f887a4"
format_stamp: "Formatted at 2026-08-06 20:30:42 on dist-test-slave-6k22"
server_key: "ee3088cb39c2595b248bebbb399a522f"
server_key_iv: "8cb1afe58101bf806d35eaf4af5efc3a"
server_key_version: "encryptionkey@0"
I20260806 20:30:43.000348 8774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:43.041894 8774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:43.046402 8774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:43.046718 8774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:43.066488 8774 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:44885
I20260806 20:30:43.066509 8848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:44885 every 8 connection(s)
I20260806 20:30:43.067991 8774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:43.070914 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 8774
I20260806 20:30:43.071118 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:43.071707 17847 external_mini_cluster.cc:1428] Setting key c41aa2e113e873710ea1c19113b07805
I20260806 20:30:43.072160 8849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:43.086246 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:43.073567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57973 (local address 127.17.109.254:44885)
0806 20:30:43.074427 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:43.074440 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:43.074482 (+ 42us) server_negotiation.cc:407] Connection header received
0806 20:30:43.076927 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:43.076949 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:43.077454 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:43.077806 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:43.078438 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:43.079865 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:43.081704 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:43.082386 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:43.082512 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:43.083903 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:43.083911 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:43.083930 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:43.084499 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:43.084582 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:43.084606 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:43.084727 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:43.085022 (+ 295us) server_negotiation.cc:299] Negotiation successful
0806 20:30:43.085236 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":400,"threads_started":1}
I20260806 20:30:43.086637 8849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4: Bootstrap starting.
I20260806 20:30:43.089936 8849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:43.090216 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:43.091241 8849 log.cc:824] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:43.094887 8849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4: No bootstrap required, opened a new log
I20260806 20:30:43.102035 8852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:43.095537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38359 (local address 127.17.109.254:44885)
0806 20:30:43.095811 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:43.095820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:43.096232 (+ 412us) server_negotiation.cc:407] Connection header received
0806 20:30:43.096472 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:43.096476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:43.096536 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:43.096642 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:43.097167 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:43.097961 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:43.099310 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:43.099861 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:43.100026 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:43.101254 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:43.101259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:43.101262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:43.101282 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:43.101333 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:43.101348 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:43.101480 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:43.101690 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 20:30:43.101758 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 20:30:43.103258 8849 raft_consensus.cc:348] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab883e8e6314a419602231e30f887a4" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 44885 } }
I20260806 20:30:43.103757 8849 raft_consensus.cc:374] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:43.103848 8849 raft_consensus.cc:729] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab883e8e6314a419602231e30f887a4, State: Initialized, Role: FOLLOWER
I20260806 20:30:43.104409 8849 consensus_queue.cc:260] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab883e8e6314a419602231e30f887a4" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 44885 } }
I20260806 20:30:43.104672 8849 raft_consensus.cc:388] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:43.104827 8849 raft_consensus.cc:482] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:43.105002 8849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:43.107527 8849 raft_consensus.cc:504] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab883e8e6314a419602231e30f887a4" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 44885 } }
I20260806 20:30:43.108184 8849 leader_election.cc:302] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cab883e8e6314a419602231e30f887a4; no voters:
I20260806 20:30:43.108764 8849 leader_election.cc:288] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:43.108939 8859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:43.109632 8859 raft_consensus.cc:686] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [term 1 LEADER]: Becoming Leader. State: Replica: cab883e8e6314a419602231e30f887a4, State: Running, Role: LEADER
I20260806 20:30:43.110208 8859 consensus_queue.cc:237] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab883e8e6314a419602231e30f887a4" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 44885 } }
I20260806 20:30:43.110952 8849 sys_catalog.cc:564] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:43.113229 8861 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cab883e8e6314a419602231e30f887a4. Latest consensus state: current_term: 1 leader_uuid: "cab883e8e6314a419602231e30f887a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab883e8e6314a419602231e30f887a4" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 44885 } } }
I20260806 20:30:43.113634 8861 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:43.113353 8860 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cab883e8e6314a419602231e30f887a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab883e8e6314a419602231e30f887a4" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 44885 } } }
I20260806 20:30:43.113765 8860 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:43.117355 8864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:43.122694 8864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:43.132695 8864 catalog_manager.cc:1347] Generated new cluster ID: 690be4189df440d9be299d7628081d58
I20260806 20:30:43.132836 8864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:43.158365 8864 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:43.160199 8864 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:43.167302 8864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cab883e8e6314a419602231e30f887a4: Generated new TSK 0
I20260806 20:30:43.168184 8864 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:43.175838 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8774
W20260806 20:30:43.177074 8844 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57973
W20260806 20:30:43.177873 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8774: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:43.178031 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 8774
W20260806 20:30:43.197614 8759 connection.cc:535] client connection to 127.17.109.254:44885 recv error: Network error: failed to read from TLS socket (remote: 127.17.109.254:44885): Connection reset by peer (error 104)
2026-08-06T20:30:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2087 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:43 dist-test-slave-6k22 krb5kdc[8883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:43 dist-test-slave-6k22 krb5kdc[8883](info): set up 2 sockets
Aug 06 20:30:43 dist-test-slave-6k22 krb5kdc[8883](info): commencing operation
krb5kdc: starting...
W20260806 20:30:44.385336 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:44 dist-test-slave-6k22 krb5kdc[8883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:44.723948 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46087
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:34687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:45.078583 8897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:45.078912 8897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:45.079098 8897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:45.086815 8897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:45.086943 8897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:45.087093 8897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:45.087216 8897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:45.087311 8897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:45.097826 8897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:34687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:45.099315 8897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:45.100822 8897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:45.113926 8906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:45.114050 8903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:45.114056 8904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:45.115393 8897 server_base.cc:1034] running on GCE node
I20260806 20:30:45.116236 8897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:45.117841 8897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:45.119094 8897 hybrid_clock.cc:648] HybridClock initialized: now 1786048245119043 us; error 50 us; skew 500 ppm
I20260806 20:30:45.119468 8897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:45 dist-test-slave-6k22 krb5kdc[8883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048245, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:45.122627 8897 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:45.124248 8897 webserver.cc:460] Webserver started at http://127.17.109.254:38069/ using document root <none> and password file <none>
I20260806 20:30:45.124948 8897 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:45.125105 8897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:45.125423 8897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:45.128479 8897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "26f1f6143d924690b84952efb7342987"
format_stamp: "Formatted at 2026-08-06 20:30:45 on dist-test-slave-6k22"
server_key: "3dd8ac6f2d124e98889b19f820479860"
server_key_iv: "09167e9acf39b922b4175ab860b0b510"
server_key_version: "encryptionkey@0"
I20260806 20:30:45.129247 8897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "26f1f6143d924690b84952efb7342987"
format_stamp: "Formatted at 2026-08-06 20:30:45 on dist-test-slave-6k22"
server_key: "3dd8ac6f2d124e98889b19f820479860"
server_key_iv: "09167e9acf39b922b4175ab860b0b510"
server_key_version: "encryptionkey@0"
I20260806 20:30:45.134459 8897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 20:30:45.137954 8915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:45.139339 8897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 20:30:45.139529 8897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "26f1f6143d924690b84952efb7342987"
format_stamp: "Formatted at 2026-08-06 20:30:45 on dist-test-slave-6k22"
server_key: "3dd8ac6f2d124e98889b19f820479860"
server_key_iv: "09167e9acf39b922b4175ab860b0b510"
server_key_version: "encryptionkey@0"
I20260806 20:30:45.139803 8897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:45.145479 8897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:45.149529 8897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:45.149824 8897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:45.168677 8897 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:46087
I20260806 20:30:45.168721 8970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:46087 every 8 connection(s)
I20260806 20:30:45.170054 8897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:45.173794 8971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:45.177258 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 8897
I20260806 20:30:45.177444 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:45.177990 17847 external_mini_cluster.cc:1428] Setting key 17f28645073864b2a2b133d20a6db24a
I20260806 20:30:45.185745 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:45.179900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48063 (local address 127.17.109.254:46087)
0806 20:30:45.180661 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:45.180672 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:45.180712 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:30:45.182516 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:45.182537 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:45.182881 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:45.183211 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:45.184265 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:45.184271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:45.184283 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:45.184303 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:45.184364 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:45.184383 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:45.184562 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:45.184667 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 20:30:45.184871 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":357,"threads_started":1}
I20260806 20:30:45.186676 8971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987: Bootstrap starting.
I20260806 20:30:45.189409 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:45.190073 8971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:45.191105 8971 log.cc:824] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:45.193758 8971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987: No bootstrap required, opened a new log
I20260806 20:30:45.196424 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:45.193898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46973 (local address 127.17.109.254:46087)
0806 20:30:45.194074 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:45.194080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:45.194560 (+ 480us) server_negotiation.cc:407] Connection header received
0806 20:30:45.194682 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:45.194687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:45.194750 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:45.194855 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:45.195786 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:45.195791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:45.195796 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:45.195814 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:45.195864 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:45.195884 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:45.196011 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:45.196081 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 20:30:45.196132 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 20:30:45.201586 8971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f1f6143d924690b84952efb7342987" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46087 } }
I20260806 20:30:45.201900 8971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:45.201973 8971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f1f6143d924690b84952efb7342987, State: Initialized, Role: FOLLOWER
I20260806 20:30:45.202431 8971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f1f6143d924690b84952efb7342987" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46087 } }
I20260806 20:30:45.202622 8971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:45.202723 8971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:45.202840 8971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:45.204939 8971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f1f6143d924690b84952efb7342987" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46087 } }
I20260806 20:30:45.205426 8971 leader_election.cc:302] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26f1f6143d924690b84952efb7342987; no voters:
I20260806 20:30:45.206012 8971 leader_election.cc:288] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:45.206142 8983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:45.206845 8983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [term 1 LEADER]: Becoming Leader. State: Replica: 26f1f6143d924690b84952efb7342987, State: Running, Role: LEADER
I20260806 20:30:45.207378 8983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f1f6143d924690b84952efb7342987" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46087 } }
I20260806 20:30:45.208163 8971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:45.210234 8984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26f1f6143d924690b84952efb7342987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f1f6143d924690b84952efb7342987" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46087 } } }
I20260806 20:30:45.210562 8984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:45.211022 8985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26f1f6143d924690b84952efb7342987. Latest consensus state: current_term: 1 leader_uuid: "26f1f6143d924690b84952efb7342987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f1f6143d924690b84952efb7342987" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46087 } } }
I20260806 20:30:45.211334 8985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:45.215896 8989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:45.223078 8989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:45.232084 8989 catalog_manager.cc:1347] Generated new cluster ID: 936fb0aad9ff4193940dc2291619f115
I20260806 20:30:45.232245 8989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:45.249046 8989 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:45.249511 8989 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:45.259959 8989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26f1f6143d924690b84952efb7342987: Generated new TSK 0
I20260806 20:30:45.260851 8989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:45.270455 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8897
W20260806 20:30:45.271953 8966 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48063
W20260806 20:30:45.272696 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 8897: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:45.272878 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 8897
2026-08-06T20:30:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2096 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:45 dist-test-slave-6k22 krb5kdc[9005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:45 dist-test-slave-6k22 krb5kdc[9005](info): set up 2 sockets
Aug 06 20:30:45 dist-test-slave-6k22 krb5kdc[9005](info): commencing operation
krb5kdc: starting...
W20260806 20:30:46.460143 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:46 dist-test-slave-6k22 krb5kdc[9005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:46.794922 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:33639
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:44481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:33639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:47.144241 9019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:47.144529 9019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:47.144737 9019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:47.152117 9019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:47.152225 9019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:47.152350 9019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:47.152433 9019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:47.152508 9019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:47.162441 9019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:44481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:33639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:33639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:47.163802 9019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:47.165161 9019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:47.177138 9024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:47.177017 9027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:47.177070 9025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:47.178229 9019 server_base.cc:1034] running on GCE node
I20260806 20:30:47.179008 9019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:47.180414 9019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:47.181653 9019 hybrid_clock.cc:648] HybridClock initialized: now 1786048247181618 us; error 45 us; skew 500 ppm
I20260806 20:30:47.182008 9019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:47 dist-test-slave-6k22 krb5kdc[9005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048247, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:47.185242 9019 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:47.186836 9019 webserver.cc:460] Webserver started at http://127.17.109.254:35755/ using document root <none> and password file <none>
I20260806 20:30:47.187547 9019 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:47.187722 9019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:47.187987 9019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:47.190865 9019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "59da8bda9972448db35ca7503ccc554d"
format_stamp: "Formatted at 2026-08-06 20:30:47 on dist-test-slave-6k22"
server_key: "5d6008d1f279f7c08c285185a29a877a"
server_key_iv: "d8a33da876c10ebbcd7b7f9640ee2766"
server_key_version: "encryptionkey@0"
I20260806 20:30:47.191619 9019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "59da8bda9972448db35ca7503ccc554d"
format_stamp: "Formatted at 2026-08-06 20:30:47 on dist-test-slave-6k22"
server_key: "5d6008d1f279f7c08c285185a29a877a"
server_key_iv: "d8a33da876c10ebbcd7b7f9640ee2766"
server_key_version: "encryptionkey@0"
I20260806 20:30:47.196584 9019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260806 20:30:47.200167 9034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:47.201504 9019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:30:47.201682 9019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "59da8bda9972448db35ca7503ccc554d"
format_stamp: "Formatted at 2026-08-06 20:30:47 on dist-test-slave-6k22"
server_key: "5d6008d1f279f7c08c285185a29a877a"
server_key_iv: "d8a33da876c10ebbcd7b7f9640ee2766"
server_key_version: "encryptionkey@0"
I20260806 20:30:47.201906 9019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:47.221745 9019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:47.226143 9019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:47.226439 9019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:47.246083 9019 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:33639
I20260806 20:30:47.246102 9091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:33639 every 8 connection(s)
I20260806 20:30:47.247500 9019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:47.248854 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 9019
I20260806 20:30:47.249042 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:47.249653 17847 external_mini_cluster.cc:1428] Setting key 774a22fbd853ddeaa6027baf88b0ad50
I20260806 20:30:47.252233 9092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 06 20:30:47 dist-test-slave-6k22 krb5kdc[9005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:47.265779 9092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d: Bootstrap starting.
I20260806 20:30:47.270201 9092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:47.271499 9092 log.cc:824] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:47.274700 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:47.251508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45917 (local address 127.17.109.254:33639)
0806 20:30:47.252404 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:47.252416 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:47.252457 (+ 41us) server_negotiation.cc:407] Connection header received
0806 20:30:47.254110 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:47.254130 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:47.254518 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:47.254855 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:47.256691 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:47.258109 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:47.259898 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:47.260546 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:47.260668 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:47.265768 (+ 5100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:47.265781 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:47.265802 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:47.265863 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:47.269881 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:47.270693 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:47.270702 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:47.270710 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:47.270924 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:47.271338 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:47.271343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:47.271346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:47.272138 (+ 792us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:47.272432 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:47.272874 (+ 442us) server_negotiation.cc:299] Negotiation successful
0806 20:30:47.273376 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":375,"threads_started":1}
I20260806 20:30:47.275082 9092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d: No bootstrap required, opened a new log
I20260806 20:30:47.278561 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:47.283663 9092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59da8bda9972448db35ca7503ccc554d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33639 } }
I20260806 20:30:47.284189 9092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:47.284317 9092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59da8bda9972448db35ca7503ccc554d, State: Initialized, Role: FOLLOWER
I20260806 20:30:47.284950 9092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59da8bda9972448db35ca7503ccc554d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33639 } }
I20260806 20:30:47.285179 9092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:47.285317 9092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:47.285496 9092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:47.288453 9092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59da8bda9972448db35ca7503ccc554d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33639 } }
I20260806 20:30:47.289134 9092 leader_election.cc:302] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59da8bda9972448db35ca7503ccc554d; no voters:
I20260806 20:30:47.289822 9092 leader_election.cc:288] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:47.289966 9103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:47.291997 9092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:47.292698 9103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [term 1 LEADER]: Becoming Leader. State: Replica: 59da8bda9972448db35ca7503ccc554d, State: Running, Role: LEADER
I20260806 20:30:47.293346 9103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59da8bda9972448db35ca7503ccc554d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33639 } }
I20260806 20:30:47.296675 9105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59da8bda9972448db35ca7503ccc554d. Latest consensus state: current_term: 1 leader_uuid: "59da8bda9972448db35ca7503ccc554d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59da8bda9972448db35ca7503ccc554d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33639 } } }
I20260806 20:30:47.296661 9104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59da8bda9972448db35ca7503ccc554d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59da8bda9972448db35ca7503ccc554d" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 33639 } } }
I20260806 20:30:47.297160 9105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:47.297160 9104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:47.302263 9096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:47.282939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45557 (local address 127.17.109.254:33639)
0806 20:30:47.283126 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:47.283133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:47.283587 (+ 454us) server_negotiation.cc:407] Connection header received
0806 20:30:47.283777 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:47.283782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:47.283858 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:47.283971 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:47.285720 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:47.286879 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:47.288177 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:47.288874 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:47.289004 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:47.292432 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:47.292441 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:47.292447 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:47.292485 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:47.297456 (+ 4971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:47.298358 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:47.298364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:47.298382 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:47.298550 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:47.299017 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:47.299025 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:47.299031 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:47.299535 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:47.299921 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:47.301399 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0806 20:30:47.301692 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 20:30:47.308811 9112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:47.313920 9112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:47.322795 9112 catalog_manager.cc:1347] Generated new cluster ID: bf3938322c464d7a899bff9a246f3afb
I20260806 20:30:47.322921 9112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:47.331496 9112 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:47.333310 9112 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:47.343685 9112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59da8bda9972448db35ca7503ccc554d: Generated new TSK 0
I20260806 20:30:47.344561 9112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:47.357952 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9019
I20260806 20:30:47.359937 9087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:47.450994 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 9019
2026-08-06T20:30:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:47 dist-test-slave-6k22 krb5kdc[9125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:47 dist-test-slave-6k22 krb5kdc[9125](info): set up 2 sockets
Aug 06 20:30:47 dist-test-slave-6k22 krb5kdc[9125](info): commencing operation
krb5kdc: starting...
W20260806 20:30:48.649881 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:48 dist-test-slave-6k22 krb5kdc[9125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:48.999075 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:45451
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:37507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:45451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:49.348680 9139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:49.349012 9139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:49.349112 9139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:49.356658 9139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:49.356807 9139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:49.356858 9139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:49.356922 9139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:49.356967 9139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:49.367316 9139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:37507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:45451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:45451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:49.368803 9139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:49.370158 9139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:49.382620 9145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:49.382602 9144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:49.382763 9147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:49.383529 9139 server_base.cc:1034] running on GCE node
I20260806 20:30:49.384440 9139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:49.385977 9139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:49.387221 9139 hybrid_clock.cc:648] HybridClock initialized: now 1786048249387148 us; error 58 us; skew 500 ppm
I20260806 20:30:49.387705 9139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:49 dist-test-slave-6k22 krb5kdc[9125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048249, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:49.391261 9139 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:49.392971 9139 webserver.cc:460] Webserver started at http://127.17.109.254:36471/ using document root <none> and password file <none>
I20260806 20:30:49.393678 9139 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:49.393788 9139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:49.394068 9139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:49.397747 9139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "06fa9878664b4650a7cc0a22fe7b2283"
format_stamp: "Formatted at 2026-08-06 20:30:49 on dist-test-slave-6k22"
server_key: "dbde2631352efadda7489eb1714f413e"
server_key_iv: "fe489d6665b9bf4e24f1274af303c9e3"
server_key_version: "encryptionkey@0"
I20260806 20:30:49.398512 9139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "06fa9878664b4650a7cc0a22fe7b2283"
format_stamp: "Formatted at 2026-08-06 20:30:49 on dist-test-slave-6k22"
server_key: "dbde2631352efadda7489eb1714f413e"
server_key_iv: "fe489d6665b9bf4e24f1274af303c9e3"
server_key_version: "encryptionkey@0"
I20260806 20:30:49.404012 9139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260806 20:30:49.407760 9154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:49.409319 9139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:49.409549 9139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "06fa9878664b4650a7cc0a22fe7b2283"
format_stamp: "Formatted at 2026-08-06 20:30:49 on dist-test-slave-6k22"
server_key: "dbde2631352efadda7489eb1714f413e"
server_key_iv: "fe489d6665b9bf4e24f1274af303c9e3"
server_key_version: "encryptionkey@0"
I20260806 20:30:49.409791 9139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:49.424871 9139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:49.429737 9139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:49.430048 9139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:49.449256 9139 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:45451
I20260806 20:30:49.449280 9211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:45451 every 8 connection(s)
I20260806 20:30:49.450747 9139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:49.453720 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 9139
I20260806 20:30:49.453924 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:49.454497 17847 external_mini_cluster.cc:1428] Setting key f1f40c1b1f04d0f78d62b49b5b656b14
I20260806 20:30:49.455107 9212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:49.469939 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:49.456487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34377 (local address 127.17.109.254:45451)
0806 20:30:49.457306 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:49.457324 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:49.457374 (+ 50us) server_negotiation.cc:407] Connection header received
0806 20:30:49.459153 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:49.459182 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:49.459789 (+ 607us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:49.460265 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:49.460794 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:49.462884 (+ 2090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:49.464372 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:49.465453 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:49.465643 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:49.466877 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:49.466888 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:49.466907 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:49.467739 (+ 832us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:49.467844 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:49.467878 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:49.468005 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:49.468243 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 20:30:49.468561 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":353,"threads_started":1}
I20260806 20:30:49.471583 9212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283: Bootstrap starting.
I20260806 20:30:49.474033 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:49.475113 9212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:49.476250 9212 log.cc:824] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:49.479839 9212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283: No bootstrap required, opened a new log
I20260806 20:30:49.484956 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:49.478697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48595 (local address 127.17.109.254:45451)
0806 20:30:49.478899 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:49.478907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:49.479299 (+ 392us) server_negotiation.cc:407] Connection header received
0806 20:30:49.479521 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:49.479525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:49.479643 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:49.479747 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:49.480307 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:49.481055 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:49.482389 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:49.482868 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:49.483010 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:49.484169 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:49.484185 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:49.484189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:49.484206 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:49.484243 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:49.484261 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:49.484388 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:49.484527 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 20:30:49.484627 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 20:30:49.488477 9212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fa9878664b4650a7cc0a22fe7b2283" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45451 } }
I20260806 20:30:49.488898 9212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:49.489017 9212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06fa9878664b4650a7cc0a22fe7b2283, State: Initialized, Role: FOLLOWER
I20260806 20:30:49.489660 9212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fa9878664b4650a7cc0a22fe7b2283" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45451 } }
I20260806 20:30:49.489914 9212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:49.490064 9212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:49.490278 9212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:49.492738 9212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fa9878664b4650a7cc0a22fe7b2283" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45451 } }
I20260806 20:30:49.493260 9212 leader_election.cc:302] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06fa9878664b4650a7cc0a22fe7b2283; no voters:
I20260806 20:30:49.493891 9212 leader_election.cc:288] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:49.494024 9222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:49.494757 9222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [term 1 LEADER]: Becoming Leader. State: Replica: 06fa9878664b4650a7cc0a22fe7b2283, State: Running, Role: LEADER
I20260806 20:30:49.495366 9222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fa9878664b4650a7cc0a22fe7b2283" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45451 } }
I20260806 20:30:49.495999 9212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:49.498070 9223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06fa9878664b4650a7cc0a22fe7b2283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fa9878664b4650a7cc0a22fe7b2283" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45451 } } }
I20260806 20:30:49.499116 9223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:49.498730 9225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06fa9878664b4650a7cc0a22fe7b2283. Latest consensus state: current_term: 1 leader_uuid: "06fa9878664b4650a7cc0a22fe7b2283" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06fa9878664b4650a7cc0a22fe7b2283" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 45451 } } }
I20260806 20:30:49.499684 9225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:49.500746 9232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:49.507421 9232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:49.516511 9232 catalog_manager.cc:1347] Generated new cluster ID: dd5e1437c87c4694968b5c17a61da188
I20260806 20:30:49.516630 9232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:49.534298 9232 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:49.536007 9232 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:49.541325 9232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06fa9878664b4650a7cc0a22fe7b2283: Generated new TSK 0
I20260806 20:30:49.542178 9232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:49.549170 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9139
W20260806 20:30:49.550462 9207 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34377
W20260806 20:30:49.551199 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9139: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:49.551370 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 9139
2026-08-06T20:30:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2100 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:49 dist-test-slave-6k22 krb5kdc[9245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:49 dist-test-slave-6k22 krb5kdc[9245](info): set up 2 sockets
Aug 06 20:30:49 dist-test-slave-6k22 krb5kdc[9245](info): commencing operation
krb5kdc: starting...
W20260806 20:30:50.788564 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:51 dist-test-slave-6k22 krb5kdc[9245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:51.127247 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:35437
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:35987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:35437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:51.472235 9258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:51.472544 9258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:51.472720 9258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:51.480041 9258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:51.480173 9258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:51.480259 9258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:51.480342 9258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:51.480406 9258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:51.490803 9258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:35987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:35437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:51.492877 9258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:51.494444 9258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:51.507696 9264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:51.507783 9267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:51.507834 9265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:51.508818 9258 server_base.cc:1034] running on GCE node
I20260806 20:30:51.509763 9258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:51.511296 9258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:51.512547 9258 hybrid_clock.cc:648] HybridClock initialized: now 1786048251512484 us; error 69 us; skew 500 ppm
I20260806 20:30:51.512971 9258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:51 dist-test-slave-6k22 krb5kdc[9245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048251, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:51.522782 9258 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:51.524545 9258 webserver.cc:460] Webserver started at http://127.17.109.254:37421/ using document root <none> and password file <none>
I20260806 20:30:51.525306 9258 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:51.525462 9258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:51.525815 9258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:51.529182 9258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "3cbbe48b49cc46fe9248c6d169795919"
format_stamp: "Formatted at 2026-08-06 20:30:51 on dist-test-slave-6k22"
server_key: "b0bf373ab67ff701fa578b757445de99"
server_key_iv: "34def202a23fd7c8e188901bbe1e24b2"
server_key_version: "encryptionkey@0"
I20260806 20:30:51.530045 9258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "3cbbe48b49cc46fe9248c6d169795919"
format_stamp: "Formatted at 2026-08-06 20:30:51 on dist-test-slave-6k22"
server_key: "b0bf373ab67ff701fa578b757445de99"
server_key_iv: "34def202a23fd7c8e188901bbe1e24b2"
server_key_version: "encryptionkey@0"
I20260806 20:30:51.535708 9258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 20:30:51.539409 9275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:51.540808 9258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 20:30:51.541002 9258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "3cbbe48b49cc46fe9248c6d169795919"
format_stamp: "Formatted at 2026-08-06 20:30:51 on dist-test-slave-6k22"
server_key: "b0bf373ab67ff701fa578b757445de99"
server_key_iv: "34def202a23fd7c8e188901bbe1e24b2"
server_key_version: "encryptionkey@0"
I20260806 20:30:51.541231 9258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:51.547804 9258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:51.552868 9258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:51.553182 9258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:51.572479 9258 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:35437
I20260806 20:30:51.572512 9332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:35437 every 8 connection(s)
I20260806 20:30:51.573923 9258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:51.577770 9333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:51.583444 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 9258
I20260806 20:30:51.583648 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:51.584200 17847 external_mini_cluster.cc:1428] Setting key 9a951d109c55dd2bd07da15f5e6ff4b3
I20260806 20:30:51.590586 9333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919: Bootstrap starting.
I20260806 20:30:51.593282 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:51.586102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51721 (local address 127.17.109.254:35437)
0806 20:30:51.587374 (+ 1272us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:51.587394 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:51.587452 (+ 58us) server_negotiation.cc:407] Connection header received
0806 20:30:51.589720 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:51.589746 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:51.590137 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:51.590498 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:51.591635 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:51.591649 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:51.591662 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:51.591684 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:51.591742 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:51.591761 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:51.591941 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:51.592064 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 20:30:51.592305 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":998,"thread_start_us":704,"threads_started":1}
I20260806 20:30:51.594269 9333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:51.595703 9333 log.cc:824] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:51.597470 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:51.598845 9333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919: No bootstrap required, opened a new log
I20260806 20:30:51.604544 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:51.601888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45469 (local address 127.17.109.254:35437)
0806 20:30:51.602088 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:51.602097 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:51.602500 (+ 403us) server_negotiation.cc:407] Connection header received
0806 20:30:51.602702 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:51.602712 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:51.602763 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:51.602869 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:51.603860 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:51.603865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:51.603870 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 20:30:51.603889 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:51.603939 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 20:30:51.603954 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:51.604081 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:51.604182 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 20:30:51.604278 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 20:30:51.607831 9333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbbe48b49cc46fe9248c6d169795919" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35437 } }
I20260806 20:30:51.608202 9333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 20:30:51.608322 9333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cbbe48b49cc46fe9248c6d169795919, State: Initialized, Role: FOLLOWER
I20260806 20:30:51.608935 9333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbbe48b49cc46fe9248c6d169795919" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35437 } }
I20260806 20:30:51.609185 9333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 20:30:51.609338 9333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 20:30:51.609521 9333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:51.612164 9333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbbe48b49cc46fe9248c6d169795919" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35437 } }
I20260806 20:30:51.612710 9333 leader_election.cc:302] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cbbe48b49cc46fe9248c6d169795919; no voters:
I20260806 20:30:51.613260 9333 leader_election.cc:288] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 20:30:51.613406 9344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:51.614084 9344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [term 1 LEADER]: Becoming Leader. State: Replica: 3cbbe48b49cc46fe9248c6d169795919, State: Running, Role: LEADER
I20260806 20:30:51.614657 9344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbbe48b49cc46fe9248c6d169795919" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35437 } }
I20260806 20:30:51.615774 9333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:51.617612 9345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cbbe48b49cc46fe9248c6d169795919" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbbe48b49cc46fe9248c6d169795919" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35437 } } }
I20260806 20:30:51.618088 9345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:51.618660 9346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cbbe48b49cc46fe9248c6d169795919. Latest consensus state: current_term: 1 leader_uuid: "3cbbe48b49cc46fe9248c6d169795919" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbbe48b49cc46fe9248c6d169795919" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 35437 } } }
I20260806 20:30:51.619263 9346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:51.626106 9353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:51.631456 9353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:51.640736 9353 catalog_manager.cc:1347] Generated new cluster ID: 35dbdf52018d4bedb1f7bfb664e307f3
I20260806 20:30:51.640865 9353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:51.663360 9353 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:51.663836 9353 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:51.671692 9353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cbbe48b49cc46fe9248c6d169795919: Generated new TSK 0
I20260806 20:30:51.672801 9353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:51.690420 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9258
W20260806 20:30:51.691636 9327 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51721
W20260806 20:30:51.692358 17847 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9258: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260806 20:30:51.692518 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 9258
2026-08-06T20:30:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2142 ms)
[----------] 18 tests from AuthTokenIssuingTest (38957 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:51 dist-test-slave-6k22 krb5kdc[9367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:51 dist-test-slave-6k22 krb5kdc[9367](info): set up 2 sockets
Aug 06 20:30:51 dist-test-slave-6k22 krb5kdc[9367](info): commencing operation
krb5kdc: starting...
W20260806 20:30:52.873159 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:53 dist-test-slave-6k22 krb5kdc[9367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:53.207276 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34129
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34129,127.17.109.253:45377,127.17.109.252:37477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:53.556653 9384 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 20:30:53.557086 9384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:53.557224 9384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:53.557394 9384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:53.565662 9384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:53.565789 9384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:53.565893 9384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:53.565985 9384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:53.566105 9384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:53.577857 9384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34129,127.17.109.253:45377,127.17.109.252:37477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:34129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:53.579468 9384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:53.581040 9384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:53.594610 9389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:53.594686 9390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:53.595369 9392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:53.600754 9384 server_base.cc:1034] running on GCE node
I20260806 20:30:53.601687 9384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:53.603242 9384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:53.604485 9384 hybrid_clock.cc:648] HybridClock initialized: now 1786048253604381 us; error 54 us; skew 500 ppm
I20260806 20:30:53.604915 9384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:53 dist-test-slave-6k22 krb5kdc[9367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048253, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:53.608450 9384 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:53.610102 9384 webserver.cc:460] Webserver started at http://127.17.109.254:45281/ using document root <none> and password file <none>
I20260806 20:30:53.610868 9384 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:53.610962 9384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:53.611251 9384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:53.614544 9384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "f15d084463564cff8256b3a1b035e999"
format_stamp: "Formatted at 2026-08-06 20:30:53 on dist-test-slave-6k22"
server_key: "2541546b3846a2f0c56b09ef390bc90a"
server_key_iv: "50a39419c1f593a43ebc52e5b649ca5b"
server_key_version: "encryptionkey@0"
I20260806 20:30:53.615357 9384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "f15d084463564cff8256b3a1b035e999"
format_stamp: "Formatted at 2026-08-06 20:30:53 on dist-test-slave-6k22"
server_key: "2541546b3846a2f0c56b09ef390bc90a"
server_key_iv: "50a39419c1f593a43ebc52e5b649ca5b"
server_key_version: "encryptionkey@0"
I20260806 20:30:53.620999 9384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 20:30:53.624723 9405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:53.626327 9384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:53.626526 9384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "f15d084463564cff8256b3a1b035e999"
format_stamp: "Formatted at 2026-08-06 20:30:53 on dist-test-slave-6k22"
server_key: "2541546b3846a2f0c56b09ef390bc90a"
server_key_iv: "50a39419c1f593a43ebc52e5b649ca5b"
server_key_version: "encryptionkey@0"
I20260806 20:30:53.626771 9384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:53.640347 9384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:53.644577 9384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:53.644866 9384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:53.664608 9384 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:34129
I20260806 20:30:53.664630 9462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:34129 every 8 connection(s)
I20260806 20:30:53.666080 9384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:53.670198 9463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:53.670462 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 9384
I20260806 20:30:53.670686 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:53.671692 17847 external_mini_cluster.cc:1428] Setting key 0f6b7e41126c88daef4123c51321e320
I20260806 20:30:53.681173 9463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:30:53 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048253, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.109.253@KRBTEST.COM; defaulting to no policy
I20260806 20:30:53.706715 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.685418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34129 (local address 127.0.0.1:38110)
0806 20:30:53.686420 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.686493 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:53.689111 (+ 2618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:53.689967 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:53.689974 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:53.690703 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:53.691045 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:53.691063 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:53.692720 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:53.692725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:53.693839 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:53.693851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:53.694799 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:53.694805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:53.695048 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:53.695809 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:53.695831 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:53.698422 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:53.702101 (+ 3679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:53.702115 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:53.702129 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:53.702575 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:53.702987 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:53.702995 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:53.702999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:53.703165 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:53.704038 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:53.704047 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:53.704283 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 20:30:53.704534 (+ 251us) client_negotiation.cc:240] Negotiation successful
0806 20:30:53.705179 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":142,"thread_start_us":403,"threads_started":1}
I20260806 20:30:53.706734 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.685418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38110 (local address 127.17.109.254:34129)
0806 20:30:53.687373 (+ 1955us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:53.687379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:53.687420 (+ 41us) server_negotiation.cc:407] Connection header received
0806 20:30:53.689492 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:53.689499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:53.689740 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:53.689945 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:53.691272 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:53.692559 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:53.693983 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:53.694553 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:53.694756 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:53.698670 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:53.698679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:53.698693 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:53.698722 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:53.701964 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:53.702712 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:53.702725 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:53.702727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:53.702875 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:53.703276 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:53.703281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:53.703284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:53.703907 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:53.704104 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:53.704671 (+ 567us) server_negotiation.cc:299] Negotiation successful
0806 20:30:53.705056 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":1755,"spinlock_wait_cycles":17664,"thread_start_us":1518,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.253@KRBTEST.COM" created.
I20260806 20:30:53.709877 9463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:30:53.711289 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.710768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33694)
0806 20:30:53.710983 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.711126 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 20:30:53.711639 9410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
I20260806 20:30:53.713320 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.712845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33706)
0806 20:30:53.713049 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.713143 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 20:30:53.713536 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:45377: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab.
Entry for principal kudu/127.17.109.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab.
I20260806 20:30:53.750731 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.253:45377
--webserver_interface=127.17.109.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab
--principal=kudu/127.17.109.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34129,127.17.109.253:45377,127.17.109.252:37477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:30:53.762951 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } attempt: 1
I20260806 20:30:53.764246 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.763780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33716)
0806 20:30:53.763985 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.764111 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 20:30:53.765286 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.764869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33722)
0806 20:30:53.765017 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.765112 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 20:30:53.765491 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:45377: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
I20260806 20:30:53.833753 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } attempt: 2
I20260806 20:30:53.835038 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.834460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33724)
0806 20:30:53.834736 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.834862 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 20:30:53.836185 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.835760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33740)
0806 20:30:53.835928 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.836019 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 20:30:53.836401 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:45377: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
I20260806 20:30:53.927722 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } attempt: 3
I20260806 20:30:53.929082 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.928544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33742)
0806 20:30:53.928789 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.928913 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260806 20:30:53.930233 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:53.929807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33750)
0806 20:30:53.929957 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:53.930044 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 20:30:53.930490 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:45377: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
I20260806 20:30:54.073797 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } attempt: 4
I20260806 20:30:54.075315 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.074594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33764)
0806 20:30:54.074971 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.075098 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260806 20:30:54.076515 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.076063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33772)
0806 20:30:54.076240 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.076344 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 20:30:54.076754 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:45377: Network error: Client connection negotiation failed: client connection to 127.17.109.253:45377: connect: Connection refused (error 111)
W20260806 20:30:54.100231 9470 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 20:30:54.100561 9470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:54.100643 9470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:54.100726 9470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:54.108220 9470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:54.108319 9470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:54.108381 9470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:54.108424 9470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:54.108489 9470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:54.118521 9470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34129,127.17.109.253:45377,127.17.109.252:37477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab
--principal=kudu/127.17.109.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.253:45377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.109.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:54.119800 9470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:54.121047 9470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:54.132545 9478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:54.132678 9476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:54.132666 9475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:54.133667 9470 server_base.cc:1034] running on GCE node
I20260806 20:30:54.134505 9470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:54.135938 9470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:54.137176 9470 hybrid_clock.cc:648] HybridClock initialized: now 1786048254137102 us; error 51 us; skew 500 ppm
I20260806 20:30:54.137493 9470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048254, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:54.140640 9470 init.cc:377] Logged in from keytab as kudu/127.17.109.253@KRBTEST.COM (short username kudu)
I20260806 20:30:54.142077 9470 webserver.cc:460] Webserver started at http://127.17.109.253:41993/ using document root <none> and password file <none>
I20260806 20:30:54.142763 9470 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:54.142892 9470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:54.143219 9470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:54.146413 9470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/data/instance:
uuid: "24dddd9686254710bc10b60cc949ccfd"
format_stamp: "Formatted at 2026-08-06 20:30:54 on dist-test-slave-6k22"
server_key: "022ec5084f35467e2ef12148fc5c38cd"
server_key_iv: "97eab2ca0f08eb7b7d8074bb1876cdef"
server_key_version: "encryptionkey@0"
I20260806 20:30:54.147269 9470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/wal/instance:
uuid: "24dddd9686254710bc10b60cc949ccfd"
format_stamp: "Formatted at 2026-08-06 20:30:54 on dist-test-slave-6k22"
server_key: "022ec5084f35467e2ef12148fc5c38cd"
server_key_iv: "97eab2ca0f08eb7b7d8074bb1876cdef"
server_key_version: "encryptionkey@0"
I20260806 20:30:54.152495 9470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260806 20:30:54.156049 9485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:54.157402 9470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 20:30:54.157584 9470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/wal
uuid: "24dddd9686254710bc10b60cc949ccfd"
format_stamp: "Formatted at 2026-08-06 20:30:54 on dist-test-slave-6k22"
server_key: "022ec5084f35467e2ef12148fc5c38cd"
server_key_iv: "97eab2ca0f08eb7b7d8074bb1876cdef"
server_key_version: "encryptionkey@0"
I20260806 20:30:54.157812 9470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:54.178197 9470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:54.183862 9470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:54.184141 9470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:54.202979 9470 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.253:45377
I20260806 20:30:54.202982 9542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.253:45377 every 8 connection(s)
I20260806 20:30:54.204417 9470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
I20260806 20:30:54.205626 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 9470
I20260806 20:30:54.205830 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-1/wal/instance
I20260806 20:30:54.206246 17847 external_mini_cluster.cc:1428] Setting key 2804ef22651f6c5404db0b62d67612e7
I20260806 20:30:54.208261 9544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:54.214591 9544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048254, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:54.235189 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.217182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.17.109.254:34129)
0806 20:30:54.217655 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.217660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.218632 (+ 972us) server_negotiation.cc:407] Connection header received
0806 20:30:54.221202 (+ 2570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.221209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.221305 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.221431 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.222910 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.223696 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.225172 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.225586 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.225719 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.229191 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.229197 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.229201 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.229248 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.232253 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.232959 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.232966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.232970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.233094 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.233499 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.233505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.233508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.233778 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.233945 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.234597 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.234815 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":266,"threads_started":1}
I20260806 20:30:54.236219 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.217709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34129 (local address 127.0.0.1:38114)
0806 20:30:54.218469 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.218507 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:54.220902 (+ 2395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:54.221665 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:54.221673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:54.222167 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:54.222705 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.222720 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.223859 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.223863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.225021 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.225033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.225811 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.225827 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.226046 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.226793 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:54.226821 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:54.228971 (+ 2150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:54.232402 (+ 3431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.232410 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.232425 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.232844 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.233202 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.233207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.233211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.233389 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.233902 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:54.233909 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:54.234242 (+ 333us) client_negotiation.cc:769] Sending connection context
0806 20:30:54.234504 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 20:30:54.235019 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":303,"threads_started":1}
WARNING: no policy specified for kudu/127.17.109.252@KRBTEST.COM; defaulting to no policy
I20260806 20:30:54.238201 9544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.252@KRBTEST.COM" created.
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048254, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
I20260806 20:30:54.252496 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.239022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33778)
0806 20:30:54.239204 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.239225 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:54.239303 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:54.240247 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:54.240251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:54.240704 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:54.240824 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.240832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.242149 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.242157 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.242967 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.242977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.243788 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.243794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.243986 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.244537 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:54.244552 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:54.246890 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:54.249954 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.249962 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.249969 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.250551 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.250926 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.250933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.250939 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.251104 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.251724 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:54.251732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:54.251904 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 20:30:54.252046 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 20:30:54.252232 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 20:30:54.252977 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.239072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33778 (local address 127.17.109.253:45377)
0806 20:30:54.239874 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.239880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.239905 (+ 25us) server_negotiation.cc:407] Connection header received
0806 20:30:54.239944 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.239950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.240110 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.240262 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.240989 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.242024 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.243092 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.243651 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.243770 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.247089 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.247100 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.247112 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.247140 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.249804 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.250666 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.250672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.250674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.250780 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.251204 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.251207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.251209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.251516 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.251742 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.252228 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.252509 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":442,"threads_started":1}
I20260806 20:30:54.255322 9544 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:30:54.256613 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.256068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60284)
0806 20:30:54.256258 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.256403 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 20:30:54.256935 9488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
I20260806 20:30:54.258592 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.258130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60290)
0806 20:30:54.258303 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.258439 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 20:30:54.258862 9544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab.
Entry for principal kudu/127.17.109.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab.
I20260806 20:30:54.277176 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.252:37477
--webserver_interface=127.17.109.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab
--principal=kudu/127.17.109.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:46005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:34129,127.17.109.253:45377,127.17.109.252:37477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:30:54.308216 9544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 1
I20260806 20:30:54.309549 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.308985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60296)
0806 20:30:54.309226 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.309360 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260806 20:30:54.310618 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.310234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60312)
0806 20:30:54.310379 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.310486 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 20:30:54.310878 9544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
I20260806 20:30:54.376030 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } attempt: 5
I20260806 20:30:54.379105 9544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 2
I20260806 20:30:54.380163 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.379693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60326)
0806 20:30:54.379877 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.379980 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 20:30:54.381322 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.380955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60334)
0806 20:30:54.381119 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.381191 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 20:30:54.381595 9544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048253, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
I20260806 20:30:54.391919 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.376838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33782)
0806 20:30:54.377069 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.377097 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:54.377209 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:54.377586 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:54.377590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:54.378002 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:54.378186 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.378195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.379719 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.379726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.380862 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.380880 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.382004 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.382008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.382182 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.382838 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:54.382856 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:54.384979 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:54.389401 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.389407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.389410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.389816 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.390295 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.390298 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.390301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.390434 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.391128 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:54.391131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:54.391275 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 20:30:54.391391 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 20:30:54.391675 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260806 20:30:54.392495 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.376970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.17.109.253:45377)
0806 20:30:54.377191 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.377198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.377229 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:30:54.377313 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.377319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.377432 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.377543 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.378347 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.379542 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.381052 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.381828 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.381960 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.385194 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.385203 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.385209 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.385256 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.389242 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.389952 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.389959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.389963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.390161 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.390548 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.390554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.390558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.391001 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.391207 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.391547 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.391842 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 20:30:54.393577 9463 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:30:54.394582 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.394193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60336)
0806 20:30:54.394374 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.394457 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 20:30:54.395447 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.395130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60340)
0806 20:30:54.395264 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.395348 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 20:30:54.395653 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
I20260806 20:30:54.446887 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 1
I20260806 20:30:54.448163 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.447707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60344)
0806 20:30:54.447904 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.447996 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260806 20:30:54.449148 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.448829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60360)
0806 20:30:54.448969 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.449045 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 20:30:54.449393 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
I20260806 20:30:54.472887 9544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 3
I20260806 20:30:54.474196 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.473630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60372)
0806 20:30:54.473905 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.474023 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 20:30:54.475419 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.474955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60376)
0806 20:30:54.475206 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.475307 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260806 20:30:54.475670 9544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
I20260806 20:30:54.517608 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 2
I20260806 20:30:54.519035 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.518500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60384)
0806 20:30:54.518764 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.518864 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 20:30:54.520077 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.519731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60398)
0806 20:30:54.519885 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.519961 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 20:30:54.520310 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
I20260806 20:30:54.618896 9544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 4
I20260806 20:30:54.620133 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.619633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60406)
0806 20:30:54.619871 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.619973 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 20:30:54.621210 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.620876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60418)
0806 20:30:54.621033 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.621108 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 20:30:54.621428 9544 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
I20260806 20:30:54.626513 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 3
I20260806 20:30:54.627696 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.627110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60432)
0806 20:30:54.627298 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.627443 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 20:30:54.628782 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.628359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60444)
0806 20:30:54.628584 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.628669 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260806 20:30:54.628980 9463 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:37477: Network error: Client connection negotiation failed: client connection to 127.17.109.252:37477: connect: Connection refused (error 111)
W20260806 20:30:54.637182 9552 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 20:30:54.637558 9552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:54.637686 9552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:54.637822 9552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:54.645346 9552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:54.645474 9552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:54.645597 9552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:54.645690 9552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:54.645797 9552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:54.656272 9552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:46005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:34129,127.17.109.253:45377,127.17.109.252:37477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab
--principal=kudu/127.17.109.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.252:37477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.109.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:54.657656 9552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:54.659289 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:54.672181 9557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:54.672822 9558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:54.673714 9560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:54.673779 9552 server_base.cc:1034] running on GCE node
I20260806 20:30:54.674657 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:54.676092 9552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:54.677349 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786048254677274 us; error 38 us; skew 500 ppm
I20260806 20:30:54.677721 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048254, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:54.680965 9552 init.cc:377] Logged in from keytab as kudu/127.17.109.252@KRBTEST.COM (short username kudu)
I20260806 20:30:54.682446 9552 webserver.cc:460] Webserver started at http://127.17.109.252:46085/ using document root <none> and password file <none>
I20260806 20:30:54.683152 9552 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:54.683280 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:54.683650 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:54.686686 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/data/instance:
uuid: "2705718d560e4365bd9fc19361bffa09"
format_stamp: "Formatted at 2026-08-06 20:30:54 on dist-test-slave-6k22"
server_key: "39fdc9c14cb28d4ca4e57c06e67ff63b"
server_key_iv: "14017b8f12edf423268c5835cbce311e"
server_key_version: "encryptionkey@0"
I20260806 20:30:54.687503 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/wal/instance:
uuid: "2705718d560e4365bd9fc19361bffa09"
format_stamp: "Formatted at 2026-08-06 20:30:54 on dist-test-slave-6k22"
server_key: "39fdc9c14cb28d4ca4e57c06e67ff63b"
server_key_iv: "14017b8f12edf423268c5835cbce311e"
server_key_version: "encryptionkey@0"
I20260806 20:30:54.692843 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:30:54.696404 9569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:54.697674 9552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:54.697880 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/wal
uuid: "2705718d560e4365bd9fc19361bffa09"
format_stamp: "Formatted at 2026-08-06 20:30:54 on dist-test-slave-6k22"
server_key: "39fdc9c14cb28d4ca4e57c06e67ff63b"
server_key_iv: "14017b8f12edf423268c5835cbce311e"
server_key_version: "encryptionkey@0"
I20260806 20:30:54.698097 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:54.719704 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:54.723737 9552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:54.724025 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:54.742034 9552 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.252:37477
I20260806 20:30:54.742054 9625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.252:37477 every 8 connection(s)
I20260806 20:30:54.743407 9552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
I20260806 20:30:54.746999 9626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:54.749835 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 9552
I20260806 20:30:54.750029 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786047878953513-17847-0/minicluster-data/master-2/wal/instance
I20260806 20:30:54.750551 17847 external_mini_cluster.cc:1428] Setting key 13d7e3eb6698a7668ecf562ccc55dc11
I20260806 20:30:54.755326 9626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048254, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:54.774076 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.757740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.17.109.254:34129)
0806 20:30:54.758345 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.758352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.758760 (+ 408us) server_negotiation.cc:407] Connection header received
0806 20:30:54.758993 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.758999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.759079 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.759191 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.760408 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.761186 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.763005 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.763419 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.763527 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.767441 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.767451 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.767455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.767512 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.770879 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.771697 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.771701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.771703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.771826 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.772260 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.772265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.772268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.772600 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.772756 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.773451 (+ 695us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.773678 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":312,"threads_started":1}
I20260806 20:30:54.775036 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.757804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:34129 (local address 127.0.0.1:38118)
0806 20:30:54.758618 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.758658 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:54.758843 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:54.759258 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:54.759265 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:54.760014 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:54.760257 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.760266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.761343 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.761349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.762849 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.762864 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.763678 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.763687 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.763932 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.764563 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:54.764599 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:54.767210 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:54.771003 (+ 3793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.771017 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.771031 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.771592 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.771945 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.771950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.771958 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.772137 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.772773 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:54.772799 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:54.773049 (+ 250us) client_negotiation.cc:769] Sending connection context
0806 20:30:54.773305 (+ 256us) client_negotiation.cc:240] Negotiation successful
0806 20:30:54.773985 (+ 680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":201,"thread_start_us":462,"threads_started":1}
I20260806 20:30:54.775070 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.752606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.17.109.252:37477)
0806 20:30:54.753571 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.753583 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.753623 (+ 40us) server_negotiation.cc:407] Connection header received
0806 20:30:54.755252 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.755273 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.755726 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.756104 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.757613 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.759616 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.761054 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.761830 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.762021 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.766950 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.766958 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.766971 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.767004 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.770555 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.771316 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.771325 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.771328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.771485 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.771937 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.771943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.771947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.772633 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.772928 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.773290 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.773852 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":766,"thread_start_us":521,"threads_started":1}
I20260806 20:30:54.776666 9626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048254, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:54.791656 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.777490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:45377 (local address 127.0.0.1:33788)
0806 20:30:54.777783 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.777801 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:54.777922 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:54.778260 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:54.778264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:54.778796 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:54.778986 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.779000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.780437 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.780441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.781869 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.781880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.782814 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.782818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.782967 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.783542 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:54.783601 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:54.785812 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:54.789167 (+ 3355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.789172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.789176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.789706 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.790094 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.790100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.790107 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.790281 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.790912 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:54.790916 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:54.791049 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 20:30:54.791160 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 20:30:54.791358 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180}
I20260806 20:30:54.791961 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.777431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33788 (local address 127.17.109.253:45377)
0806 20:30:54.777640 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.777645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.777903 (+ 258us) server_negotiation.cc:407] Connection header received
0806 20:30:54.778042 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.778048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.778135 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.778235 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.779130 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.780289 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.781992 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.782671 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.782798 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.786005 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.786015 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.786023 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.786071 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.789019 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.789827 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.789831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.789833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.789966 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.790406 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.790412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.790416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.790778 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.790942 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.791289 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.791531 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 20:30:54.792932 9626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:30:54.794649 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.778244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.17.109.254:34129)
0806 20:30:54.778802 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.778807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.778824 (+ 17us) server_negotiation.cc:407] Connection header received
0806 20:30:54.778874 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.778878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.778961 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.779069 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.780592 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.781676 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.783266 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.783806 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.783903 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.788354 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.788359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.788362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.788393 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.791858 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.792655 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.792661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.792665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.792867 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.793255 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.793259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.793261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.793529 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.793846 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.794129 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.794360 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437}
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048254, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
I20260806 20:30:54.806190 9463 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 4
I20260806 20:30:54.807365 9632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.793801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60448)
0806 20:30:54.794013 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.794027 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:54.794139 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:54.794436 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:54.794440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:54.795026 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:54.795228 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.795242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.796489 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.796494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.797932 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.797947 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.798778 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.798782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.798995 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.799600 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:54.799626 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:54.801769 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:54.804828 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.804834 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.804838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.805194 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.805535 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.805543 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.805548 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.805705 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.806207 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:54.806211 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:54.806423 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 20:30:54.806622 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 20:30:54.806941 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":2}
I20260806 20:30:54.807986 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.793745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.17.109.252:37477)
0806 20:30:54.793912 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.793918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.794089 (+ 171us) server_negotiation.cc:407] Connection header received
0806 20:30:54.794254 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.794259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.794325 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.794425 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.795386 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.796365 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.798071 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.798638 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.798781 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.801967 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.801971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.801974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.802020 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.804704 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.805312 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.805317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.805325 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.805436 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.805826 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.805830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.805832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.806091 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.806245 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.806730 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.807446 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":451,"server-negotiator.queue_time_us":64}
I20260806 20:30:54.813158 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.796462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.17.109.253:45377)
0806 20:30:54.796608 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.796612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.796625 (+ 13us) server_negotiation.cc:407] Connection header received
0806 20:30:54.796664 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.796667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.796716 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.796795 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.798338 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.799287 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.800559 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.801006 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.801099 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.805765 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.805773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.805779 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.805823 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.809407 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.810243 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.810247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.810251 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.810429 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.810879 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.810884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.810887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.811225 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.811505 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.812578 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.812814 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 20:30:54.815248 9626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09: Bootstrap starting.
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048253, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
I20260806 20:30:54.819721 9626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:54.821420 9626 log.cc:824] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:54.824990 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:54.828872 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.806843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60462)
0806 20:30:54.807021 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.807041 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:54.807148 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:54.809133 (+ 1985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:54.809141 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:54.809841 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:54.810129 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.810147 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.812285 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.812293 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.814296 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.814318 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.815832 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.815839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.816055 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.816808 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:54.816833 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:54.819785 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:54.824835 (+ 5050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.824845 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.824852 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.825548 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.826139 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.826147 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.826153 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.826401 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.827347 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:54.827356 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:54.827652 (+ 296us) client_negotiation.cc:769] Sending connection context
0806 20:30:54.827834 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 20:30:54.828251 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 20:30:54.830350 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.807187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.17.109.252:37477)
0806 20:30:54.808720 (+ 1533us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.808728 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.808759 (+ 31us) server_negotiation.cc:407] Connection header received
0806 20:30:54.808815 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.808823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.808909 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.810318 (+ 1409us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.810493 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.812059 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.814546 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.815640 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.815822 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.820006 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.820013 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.820018 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.820055 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.824618 (+ 4563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.825759 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.825768 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.825773 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.825974 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.826577 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.826585 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.826591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.827153 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.829187 (+ 2034us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.829398 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.829717 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1397,"thread_start_us":13014,"threads_started":1}
I20260806 20:30:54.829011 9435 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f15d084463564cff8256b3a1b035e999"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38120
I20260806 20:30:54.837653 9626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09: No bootstrap required, opened a new log
I20260806 20:30:54.839913 9463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999: Bootstrap starting.
I20260806 20:30:54.844090 9463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:54.845484 9463 log.cc:824] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:54.848389 9626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } }
I20260806 20:30:54.848850 9626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2705718d560e4365bd9fc19361bffa09, State: Initialized, Role: FOLLOWER
I20260806 20:30:54.849014 9463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999: No bootstrap required, opened a new log
I20260806 20:30:54.849375 9626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } }
I20260806 20:30:54.850428 9636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:54.850898 9636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:54.851715 9626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:54.857430 9463 raft_consensus.cc:348] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } }
I20260806 20:30:54.857879 9463 raft_consensus.cc:729] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f15d084463564cff8256b3a1b035e999, State: Initialized, Role: FOLLOWER
I20260806 20:30:54.858400 9463 consensus_queue.cc:260] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } }
I20260806 20:30:54.859398 9650 sys_catalog.cc:455] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:54.860208 9650 sys_catalog.cc:458] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:54.860695 9463 sys_catalog.cc:564] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:54.867965 9435 raft_consensus.cc:482] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 20:30:54.868181 9435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:54.871744 9435 raft_consensus.cc:504] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } }
W20260806 20:30:54.872481 9649 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 20:30:54.872587 9649 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 20:30:54.873713 9435 leader_election.cc:288] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [CANDIDATE]: Term 1 election: Requested vote from peers 24dddd9686254710bc10b60cc949ccfd (127.17.109.253:45377), 2705718d560e4365bd9fc19361bffa09 (127.17.109.252:37477)
I20260806 20:30:54.874677 9598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f15d084463564cff8256b3a1b035e999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2705718d560e4365bd9fc19361bffa09"
I20260806 20:30:54.875201 9598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:54.878925 9598 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f15d084463564cff8256b3a1b035e999 in term 1.
I20260806 20:30:54.879606 9408 leader_election.cc:302] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2705718d560e4365bd9fc19361bffa09, f15d084463564cff8256b3a1b035e999; no voters:
I20260806 20:30:54.880052 9650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 20:30:54.880808 9650 raft_consensus.cc:686] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [term 1 LEADER]: Becoming Leader. State: Replica: f15d084463564cff8256b3a1b035e999, State: Running, Role: LEADER
I20260806 20:30:54.859972 9552 master_runner.cc:186] Error getting master registration for 127.17.109.254:34129: OK, instance_id { permanent_uuid: "f15d084463564cff8256b3a1b035e999" instance_seqno: 1786048253658952 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 20:30:54.881506 9552 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 20:30:54.881382 9650 consensus_queue.cc:237] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } }
I20260806 20:30:54.870455 9384 master_runner.cc:186] Error getting master registration for 127.17.109.253:45377: OK, instance_id { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" instance_seqno: 1786048254197561 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 20:30:54.884733 9384 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 20:30:54.886025 9666 sys_catalog.cc:455] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f15d084463564cff8256b3a1b035e999. Latest consensus state: current_term: 1 leader_uuid: "f15d084463564cff8256b3a1b035e999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:54.886606 9666 sys_catalog.cc:458] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: This master's current role is: LEADER
W20260806 20:30:54.894179 9662 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 20:30:54.894323 9662 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 20:30:54.894512 9668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:54.896728 9668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260806 20:30:54.897786 9413 tablet.cc:2369] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:54.920620 9544 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } attempt: 5
I20260806 20:30:54.923489 9598 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [term 1 FOLLOWER]: Refusing update from remote peer f15d084463564cff8256b3a1b035e999: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:54.927136 9666 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 06 20:30:54 dist-test-slave-6k22 krb5kdc[9367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048254, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
I20260806 20:30:54.951655 9547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.921340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:37477 (local address 127.0.0.1:60474)
0806 20:30:54.922104 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:54.922132 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:54.922278 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:54.925897 (+ 3619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:54.925906 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:54.926614 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:54.926881 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.926893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.929251 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.929256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.930618 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:54.930632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.931933 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.931939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:54.932187 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.933012 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:54.933058 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:54.937908 (+ 4850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:54.945346 (+ 7438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.945351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.945355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.946161 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.946996 (+ 835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:54.947004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:54.947010 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:54.947393 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:54.948868 (+ 1475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:54.948877 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:54.949214 (+ 337us) client_negotiation.cc:769] Sending connection context
0806 20:30:54.949851 (+ 637us) client_negotiation.cc:240] Negotiation successful
0806 20:30:54.950392 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600}
I20260806 20:30:54.952555 9636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f15d084463564cff8256b3a1b035e999. Latest consensus state: current_term: 1 leader_uuid: "f15d084463564cff8256b3a1b035e999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:54.952874 9636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:54.953524 9633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.924328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.17.109.252:37477)
0806 20:30:54.924567 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.924575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.924604 (+ 29us) server_negotiation.cc:407] Connection header received
0806 20:30:54.924672 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.924680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.924781 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.924929 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.927124 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.928359 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.930823 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.931500 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.931691 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.938321 (+ 6630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.938342 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.938350 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.938423 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.945145 (+ 6722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.946404 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.946409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.946412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.946728 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.947685 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.947693 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.947699 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.948592 (+ 893us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.950840 (+ 2248us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.951462 (+ 622us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.952606 (+ 1144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:30:54.967706 9650 sys_catalog.cc:455] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f15d084463564cff8256b3a1b035e999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:54.968091 9650 sys_catalog.cc:458] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:54.963527 9544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd: Bootstrap starting.
I20260806 20:30:54.968722 9677 mvcc.cc:205] Tried to move back new op lower bound from 7315653652147752960 to 7315653651999014912. Current Snapshot: MvccSnapshot[applied={T|T < 7315653652147752960}]
I20260806 20:30:54.977707 9544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:54.980899 9544 log.cc:824] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:54.984854 9636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f15d084463564cff8256b3a1b035e999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:54.985150 9636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:54.987828 9668 catalog_manager.cc:1347] Generated new cluster ID: a1a815a9e5d146359c194b8c2308c4f6
I20260806 20:30:54.987952 9668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:54.993570 9544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd: No bootstrap required, opened a new log
I20260806 20:30:54.996685 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.915450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60464 (local address 127.17.109.252:37477)
0806 20:30:54.915739 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.915747 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.929345 (+ 13598us) server_negotiation.cc:407] Connection header received
0806 20:30:54.930209 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.930214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.930313 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.930387 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.938217 (+ 7830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.939395 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.947726 (+ 8331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.948497 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.948599 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:54.977975 (+ 29376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:54.977981 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:54.977986 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:54.978031 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:54.982831 (+ 4800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.988706 (+ 5875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.988715 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.988721 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.988944 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:54.992145 (+ 3201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:54.992150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:54.992153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:54.992537 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:54.992798 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:54.996022 (+ 3224us) server_negotiation.cc:299] Negotiation successful
0806 20:30:54.996308 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 20:30:55.006457 9544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } }
I20260806 20:30:55.007059 9544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24dddd9686254710bc10b60cc949ccfd, State: Initialized, Role: FOLLOWER
I20260806 20:30:55.007706 9544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } }
I20260806 20:30:55.014866 9683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:55.022698 9683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:55.022555 9544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:55.022670 9516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f15d084463564cff8256b3a1b035e999" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "24dddd9686254710bc10b60cc949ccfd"
I20260806 20:30:55.023033 9515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:55.023692 9668 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:55.025876 9631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.915181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38126 (local address 127.17.109.254:34129)
0806 20:30:54.915723 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.915733 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.928495 (+ 12762us) server_negotiation.cc:407] Connection header received
0806 20:30:54.928557 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.928563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.928642 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.928737 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.962638 (+ 33901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.963913 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.984191 (+ 20278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.984898 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.985007 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.006151 (+ 21144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:55.006160 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:55.006165 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:55.006214 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:55.011069 (+ 4855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:55.014676 (+ 3607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:55.014683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:55.014686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:55.015019 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:55.016413 (+ 1394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:55.016421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:55.016452 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:55.017132 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:55.017593 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.024952 (+ 7359us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.025297 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356}
I20260806 20:30:55.025632 9668 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:55.026836 9515 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [term 1 FOLLOWER]: Refusing update from remote peer f15d084463564cff8256b3a1b035e999: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:55.027858 9650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:55.032325 9516 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f15d084463564cff8256b3a1b035e999 in term 1.
I20260806 20:30:55.045456 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:54.916377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.17.109.253:45377)
0806 20:30:54.916581 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:54.916589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:54.957661 (+ 41072us) server_negotiation.cc:407] Connection header received
0806 20:30:54.958979 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:54.958984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:54.959071 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:54.959143 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:54.973988 (+ 14845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.975273 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.995162 (+ 19889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:54.995935 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:54.996036 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.018392 (+ 22356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:55.018400 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:55.018404 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:55.018446 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:55.022457 (+ 4011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:55.040719 (+ 18262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:55.040729 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:55.040735 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:55.041008 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:55.042350 (+ 1342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:55.042358 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:55.042364 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:55.042892 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:55.043206 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.044609 (+ 1403us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.044996 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 20:30:55.055442 9668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999: Generated new TSK 0
I20260806 20:30:55.056676 9668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:55.101965 9683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f15d084463564cff8256b3a1b035e999. Latest consensus state: current_term: 1 leader_uuid: "f15d084463564cff8256b3a1b035e999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:55.102337 9683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:55.104380 9666 sys_catalog.cc:455] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f15d084463564cff8256b3a1b035e999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:55.104822 9666 sys_catalog.cc:458] T 00000000000000000000000000000000 P f15d084463564cff8256b3a1b035e999 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:55.109315 9697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f15d084463564cff8256b3a1b035e999" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f15d084463564cff8256b3a1b035e999" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 34129 } } peers { permanent_uuid: "24dddd9686254710bc10b60cc949ccfd" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 45377 } } peers { permanent_uuid: "2705718d560e4365bd9fc19361bffa09" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 37477 } } }
I20260806 20:30:55.109723 9697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:55.134850 9699 catalog_manager.cc:1259] Loaded cluster ID: a1a815a9e5d146359c194b8c2308c4f6
I20260806 20:30:55.134967 9699 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd: loading cluster ID for follower catalog manager: success
I20260806 20:30:55.138860 9699 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd: acquiring CA information for follower catalog manager: success
I20260806 20:30:55.141492 9699 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24dddd9686254710bc10b60cc949ccfd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 20:30:55.194679 17847 mini_kdc.cc:348] Time spent kdestroy: real 0.124s user 0.000s sys 0.033s
W20260806 20:30:55.254668 9631 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.228535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38136 (local address 127.17.109.254:34129)
0806 20:30:55.228785 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.228792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.251026 (+ 22234us) server_negotiation.cc:407] Connection header received
0806 20:30:55.253019 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.253026 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.253123 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.253205 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.254399 (+ 1194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38136: BlockingRecv error: recv got EOF from 127.0.0.1:38136 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 20:30:55.257084 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.235338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60478 (local address 127.17.109.252:37477)
0806 20:30:55.235513 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.235520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.252549 (+ 17029us) server_negotiation.cc:407] Connection header received
0806 20:30:55.255436 (+ 2887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.255446 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.255535 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.255631 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.256767 (+ 1136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60478: BlockingRecv error: recv got EOF from 127.0.0.1:60478 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260806 20:30:55.259918 9549 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.248532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.17.109.253:45377)
0806 20:30:55.248789 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.248796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.257843 (+ 9047us) server_negotiation.cc:407] Connection header received
0806 20:30:55.258389 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.258411 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.258517 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.258600 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.259682 (+ 1082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33816: BlockingRecv error: recv got EOF from 127.0.0.1:33816 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 20:30:55.305595 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.290250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.17.109.253:45377)
0806 20:30:55.290467 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.290473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.297330 (+ 6857us) server_negotiation.cc:407] Connection header received
0806 20:30:55.297865 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.297872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.298066 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.298192 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.298948 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.300190 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.302481 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.303209 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.303313 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.304131 (+ 818us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.304775 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.304894 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.305146 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.305245 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260806 20:30:55.356568 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.343409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60482 (local address 127.17.109.252:37477)
0806 20:30:55.343624 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.343631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.354329 (+ 10698us) server_negotiation.cc:407] Connection header received
0806 20:30:55.354903 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.354909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.355056 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.355142 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.356330 (+ 1188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60482: BlockingRecv error: recv got EOF from 127.0.0.1:60482 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 20:30:55.414513 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.399892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33844 (local address 127.17.109.253:45377)
0806 20:30:55.400088 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.400094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.407389 (+ 7295us) server_negotiation.cc:407] Connection header received
0806 20:30:55.407939 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.407946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.408075 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.408144 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.408851 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.410053 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.412167 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.412899 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.412997 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.413615 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.413783 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.413852 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.414093 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.414179 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 20:30:55.465350 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.452248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.17.109.252:37477)
0806 20:30:55.452439 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.452445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.463204 (+ 10759us) server_negotiation.cc:407] Connection header received
0806 20:30:55.463772 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.463778 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.463905 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.463985 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.465099 (+ 1114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60496: BlockingRecv error: recv got EOF from 127.0.0.1:60496 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:30:55.523241 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.501979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.17.109.253:45377)
0806 20:30:55.502237 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.502243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.516313 (+ 14070us) server_negotiation.cc:407] Connection header received
0806 20:30:55.516527 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.516533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.516628 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.516755 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.517496 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.518743 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.520861 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.521565 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.521661 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.522341 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.522497 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.522560 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.522809 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.522896 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260806 20:30:55.553714 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.548608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.17.109.252:37477)
0806 20:30:55.548794 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.548800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.552115 (+ 3315us) server_negotiation.cc:407] Connection header received
0806 20:30:55.552293 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.552298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.552397 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.552516 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.553429 (+ 913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60510: BlockingRecv error: recv got EOF from 127.0.0.1:60510 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 20:30:55.568046 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.561860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33862 (local address 127.17.109.253:45377)
0806 20:30:55.562061 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.562067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.562495 (+ 428us) server_negotiation.cc:407] Connection header received
0806 20:30:55.562707 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.562714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.562805 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.562911 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.563383 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.564613 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.565980 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.566744 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.566871 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.567252 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.567382 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.567479 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.567667 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.567753 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260806 20:30:55.576742 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.574709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.17.109.252:37477)
0806 20:30:55.574972 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.574978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.575241 (+ 263us) server_negotiation.cc:407] Connection header received
0806 20:30:55.575420 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.575423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.575497 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.575631 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.576495 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60526: BlockingRecv error: recv got EOF from 127.0.0.1:60526 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
I20260806 20:30:55.594671 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.588396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33874 (local address 127.17.109.253:45377)
0806 20:30:55.588631 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.588637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.588952 (+ 315us) server_negotiation.cc:407] Connection header received
0806 20:30:55.589161 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.589168 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.589254 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.589336 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.589806 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.590609 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.592668 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.593176 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.593279 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.593784 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.593926 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.593992 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.594207 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.594307 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260806 20:30:55.604565 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.601232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.17.109.252:37477)
0806 20:30:55.601409 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.601415 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.602584 (+ 1169us) server_negotiation.cc:407] Connection header received
0806 20:30:55.602916 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.602922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.603043 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.603119 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.604320 (+ 1201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60534: BlockingRecv error: recv got EOF from 127.0.0.1:60534 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 20:30:55.630546 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.625072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33880 (local address 127.17.109.253:45377)
0806 20:30:55.625262 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.625270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.625677 (+ 407us) server_negotiation.cc:407] Connection header received
0806 20:30:55.625928 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.625935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.626012 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.626104 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.626525 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.627361 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.628723 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.629169 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.629316 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.629741 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.629887 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.629998 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.630157 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.630258 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 20:30:55.638571 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.636452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60536 (local address 127.17.109.252:37477)
0806 20:30:55.636677 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.636683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.637044 (+ 361us) server_negotiation.cc:407] Connection header received
0806 20:30:55.637264 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.637268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.637340 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.637425 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.638317 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60536: BlockingRecv error: recv got EOF from 127.0.0.1:60536 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 20:30:55.682708 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.674985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33882 (local address 127.17.109.253:45377)
0806 20:30:55.675234 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.675242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.675932 (+ 690us) server_negotiation.cc:407] Connection header received
0806 20:30:55.676197 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.676205 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.676342 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.676441 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.677037 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.678173 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.680451 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.681180 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.681337 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.681875 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.682013 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.682079 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.682289 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.682364 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260806 20:30:55.692627 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.690453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60538 (local address 127.17.109.252:37477)
0806 20:30:55.690687 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.690692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.691015 (+ 323us) server_negotiation.cc:407] Connection header received
0806 20:30:55.691208 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.691212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.691303 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.691389 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.692407 (+ 1018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60538: BlockingRecv error: recv got EOF from 127.0.0.1:60538 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260806 20:30:55.767840 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.761069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33892 (local address 127.17.109.253:45377)
0806 20:30:55.761277 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.761284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.761868 (+ 584us) server_negotiation.cc:407] Connection header received
0806 20:30:55.762121 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.762130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.762259 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.762371 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.762919 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.764071 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.765910 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.766385 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.766487 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.767079 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.767195 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.767284 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.767429 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.767499 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 20:30:55.775988 9630 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:55.774124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60540 (local address 127.17.109.252:37477)
0806 20:30:55.774303 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.774308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.774710 (+ 402us) server_negotiation.cc:407] Connection header received
0806 20:30:55.774866 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.774870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.774941 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.775026 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:55.775807 (+ 781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60540: BlockingRecv error: recv got EOF from 127.0.0.1:60540 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260806 20:30:55.874481 9649 catalog_manager.cc:1259] Loaded cluster ID: a1a815a9e5d146359c194b8c2308c4f6
I20260806 20:30:55.874612 9649 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09: loading cluster ID for follower catalog manager: success
I20260806 20:30:55.877104 9649 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09: acquiring CA information for follower catalog manager: success
I20260806 20:30:55.878839 9649 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2705718d560e4365bd9fc19361bffa09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 20:30:55.913993 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.909271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.17.109.253:45377)
0806 20:30:55.909466 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.909470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.909810 (+ 340us) server_negotiation.cc:407] Connection header received
0806 20:30:55.909975 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.909979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.910053 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.910143 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.910532 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.911242 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.912520 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.912961 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.913053 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.913411 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.913507 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.913573 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.913702 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.913792 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 20:30:55.925334 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.919680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60552 (local address 127.17.109.252:37477)
0806 20:30:55.919924 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.919930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.920125 (+ 195us) server_negotiation.cc:407] Connection header received
0806 20:30:55.920316 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.920322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.920488 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.920607 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.920995 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.921707 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.923007 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.923499 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.923682 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.924164 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.924768 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.924933 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.924992 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.925076 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 20:30:55.938514 9630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.931712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60560 (local address 127.17.109.252:37477)
0806 20:30:55.931921 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.931929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.933043 (+ 1114us) server_negotiation.cc:407] Connection header received
0806 20:30:55.933255 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.933263 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.933362 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.933440 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.934419 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.935157 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.936724 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.937177 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.937313 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.937763 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.937865 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.937959 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.938125 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.938209 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 20:30:55.940245 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.931392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.17.109.254:34129)
0806 20:30:55.932537 (+ 1145us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.932546 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.932572 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:30:55.932704 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.932710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.932930 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.933138 (+ 208us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.933643 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.934397 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.937180 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.937862 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.937998 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.939134 (+ 1136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.939610 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.939709 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.939851 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.939939 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":857,"threads_started":1}
I20260806 20:30:55.941411 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:55.932734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.17.109.253:45377)
0806 20:30:55.932933 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:55.932939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:55.934924 (+ 1985us) server_negotiation.cc:407] Connection header received
0806 20:30:55.935077 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:55.935084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:55.935168 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:55.935257 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:30:55.936179 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.937257 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.939524 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:55.940199 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:55.940342 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:55.940745 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:30:55.940861 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:30:55.940956 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:55.941089 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 20:30:55.941166 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 20:30:55.945130 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9384
I20260806 20:30:55.946121 9457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:56.075918 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 9384
W20260806 20:30:56.100306 9572 connection.cc:535] server connection from 127.0.0.1:60462 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 20:30:56.100329 9491 connection.cc:535] server connection from 127.0.0.1:33782 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 20:30:56.100786 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9470
I20260806 20:30:56.101963 9538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:56.214699 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 9470
I20260806 20:30:56.237548 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9552
I20260806 20:30:56.238672 9621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:30:56.346138 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 9552
2026-08-06T20:30:56Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4656 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 20:30:56 dist-test-slave-6k22 krb5kdc[9828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 20:30:56 dist-test-slave-6k22 krb5kdc[9828](info): set up 2 sockets
Aug 06 20:30:56 dist-test-slave-6k22 krb5kdc[9828](info): commencing operation
krb5kdc: starting...
W20260806 20:30:57.565052 17847 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/test-user.keytab.
Aug 06 20:30:57 dist-test-slave-6k22 krb5kdc[9828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T20:30:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T20:30:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.109.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
Entry for principal kudu/127.17.109.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab.
I20260806 20:30:57.908859 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46385
--webserver_interface=127.17.109.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46385,127.17.109.253:39773,127.17.109.252:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 20:30:58.257237 9842 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 20:30:58.257611 9842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:58.257738 9842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:58.257877 9842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:58.265003 9842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:58.265080 9842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:58.265182 9842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:58.265251 9842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:58.265355 9842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:58.275121 9842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46385,127.17.109.253:39773,127.17.109.252:35091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.254.keytab
--principal=kudu/127.17.109.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.254:46385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.109.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:58.276515 9842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:58.277845 9842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:58.289927 9851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:58.290005 9847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:58.290005 9848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:58.290916 9842 server_base.cc:1034] running on GCE node
I20260806 20:30:58.291863 9842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:58.293550 9842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:58.294822 9842 hybrid_clock.cc:648] HybridClock initialized: now 1786048258294745 us; error 40 us; skew 500 ppm
I20260806 20:30:58.295220 9842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:58 dist-test-slave-6k22 krb5kdc[9828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048258, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:58.305676 9842 init.cc:377] Logged in from keytab as kudu/127.17.109.254@KRBTEST.COM (short username kudu)
I20260806 20:30:58.307301 9842 webserver.cc:460] Webserver started at http://127.17.109.254:39763/ using document root <none> and password file <none>
I20260806 20:30:58.308080 9842 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:58.308182 9842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:58.308465 9842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:58.311486 9842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/data/instance:
uuid: "5a5aef0fc58444878d4398a486c71627"
format_stamp: "Formatted at 2026-08-06 20:30:58 on dist-test-slave-6k22"
server_key: "6e2cb239350463c87e1db4692b72c833"
server_key_iv: "5ca57b78d6254d7495ffed978301b918"
server_key_version: "encryptionkey@0"
I20260806 20:30:58.312325 9842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/wal/instance:
uuid: "5a5aef0fc58444878d4398a486c71627"
format_stamp: "Formatted at 2026-08-06 20:30:58 on dist-test-slave-6k22"
server_key: "6e2cb239350463c87e1db4692b72c833"
server_key_iv: "5ca57b78d6254d7495ffed978301b918"
server_key_version: "encryptionkey@0"
I20260806 20:30:58.317709 9842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 20:30:58.321292 9859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:58.322634 9842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 20:30:58.322824 9842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/wal
uuid: "5a5aef0fc58444878d4398a486c71627"
format_stamp: "Formatted at 2026-08-06 20:30:58 on dist-test-slave-6k22"
server_key: "6e2cb239350463c87e1db4692b72c833"
server_key_iv: "5ca57b78d6254d7495ffed978301b918"
server_key_version: "encryptionkey@0"
I20260806 20:30:58.323031 9842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:58.350775 9842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:58.355221 9842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:58.355531 9842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:58.375212 9842 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.254:46385
I20260806 20:30:58.375195 9917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.254:46385 every 8 connection(s)
I20260806 20:30:58.376703 9842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/data/info.pb
I20260806 20:30:58.380710 9918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:58.382598 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 9842
I20260806 20:30:58.382814 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-0/wal/instance
I20260806 20:30:58.383504 17847 external_mini_cluster.cc:1428] Setting key 440698131f2e49e254379e430158e219
I20260806 20:30:58.390946 9918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:30:58 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048258, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:58.416451 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.395174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.17.109.254:46385)
0806 20:30:58.397129 (+ 1955us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:58.397144 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:58.397199 (+ 55us) server_negotiation.cc:407] Connection header received
0806 20:30:58.398879 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:58.398886 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:58.399131 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:58.399362 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:58.400827 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.402179 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.403790 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.404409 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.404659 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:58.408710 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:58.408718 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:58.408732 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:58.408776 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:58.411728 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:58.412712 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:58.412718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:58.412721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:58.412938 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:58.413454 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:58.413458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:58.413460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:58.413981 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:58.414142 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:58.414978 (+ 836us) server_negotiation.cc:299] Negotiation successful
0806 20:30:58.415274 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":1700,"thread_start_us":1457,"threads_started":1}
I20260806 20:30:58.416517 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.395173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46385 (local address 127.0.0.1:47088)
0806 20:30:58.396223 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.396273 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:58.398472 (+ 2199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:58.399375 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:58.399382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:58.400219 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:58.400593 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:58.400611 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.402349 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.402355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:58.403637 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:58.403654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.404673 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.404681 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:58.404877 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:58.405689 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:58.405719 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:58.408456 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:58.411873 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:58.411890 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:58.411906 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:58.412559 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:58.413083 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:58.413091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:58.413097 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:58.413327 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:58.414142 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:58.414155 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:58.414497 (+ 342us) client_negotiation.cc:769] Sending connection context
0806 20:30:58.414852 (+ 355us) client_negotiation.cc:240] Negotiation successful
0806 20:30:58.415482 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":49,"thread_start_us":474,"threads_started":1}
WARNING: no policy specified for kudu/127.17.109.253@KRBTEST.COM; defaulting to no policy
I20260806 20:30:58.419644 9918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.253@KRBTEST.COM" created.
I20260806 20:30:58.420897 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.420364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41204)
0806 20:30:58.420551 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.420744 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260806 20:30:58.421172 9864 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
I20260806 20:30:58.422888 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.422439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41218)
0806 20:30:58.422610 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.422704 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 20:30:58.423101 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:39773: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab.
Entry for principal kudu/127.17.109.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab.
I20260806 20:30:58.459296 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.253:39773
--webserver_interface=127.17.109.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab
--principal=kudu/127.17.109.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46385,127.17.109.253:39773,127.17.109.252:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:30:58.472452 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } attempt: 1
I20260806 20:30:58.473716 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.473247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41232)
0806 20:30:58.473431 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.473565 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 20:30:58.474694 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.474375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41236)
0806 20:30:58.474514 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.474582 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 20:30:58.474865 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:39773: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
I20260806 20:30:58.543124 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } attempt: 2
I20260806 20:30:58.544492 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.543860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41250)
0806 20:30:58.544170 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.544301 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 20:30:58.545759 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.545219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41254)
0806 20:30:58.545434 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.545537 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 20:30:58.546022 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:39773: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
I20260806 20:30:58.637308 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } attempt: 3
I20260806 20:30:58.638556 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.637995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41264)
0806 20:30:58.638253 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.638384 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 20:30:58.639743 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.639230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41276)
0806 20:30:58.639417 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.639522 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 20:30:58.639988 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:39773: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
I20260806 20:30:58.783288 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } attempt: 4
I20260806 20:30:58.784642 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.783980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41290)
0806 20:30:58.784360 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.784472 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260806 20:30:58.785808 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.785390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41300)
0806 20:30:58.785551 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.785654 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 20:30:58.786031 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.253:39773: Network error: Client connection negotiation failed: client connection to 127.17.109.253:39773: connect: Connection refused (error 111)
W20260806 20:30:58.800872 9930 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 20:30:58.801204 9930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:58.801278 9930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:58.801363 9930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:58.808634 9930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:58.808735 9930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:58.808795 9930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:58.808856 9930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:58.808920 9930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:58.818981 9930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46385,127.17.109.253:39773,127.17.109.252:35091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.253.keytab
--principal=kudu/127.17.109.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.253:39773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.109.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:58.820291 9930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:58.821722 9930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:58.833547 9936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:58.833662 9935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:58.833627 9938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:58.834549 9930 server_base.cc:1034] running on GCE node
I20260806 20:30:58.835330 9930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:58.836889 9930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:58.838152 9930 hybrid_clock.cc:648] HybridClock initialized: now 1786048258838065 us; error 54 us; skew 500 ppm
I20260806 20:30:58.838573 9930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:58 dist-test-slave-6k22 krb5kdc[9828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048258, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:58.842232 9930 init.cc:377] Logged in from keytab as kudu/127.17.109.253@KRBTEST.COM (short username kudu)
I20260806 20:30:58.843935 9930 webserver.cc:460] Webserver started at http://127.17.109.253:32815/ using document root <none> and password file <none>
I20260806 20:30:58.844691 9930 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:58.844798 9930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:58.845126 9930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:58.848193 9930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/data/instance:
uuid: "22fb08d988f443438a0313dbef847242"
format_stamp: "Formatted at 2026-08-06 20:30:58 on dist-test-slave-6k22"
server_key: "63edf23310269bb8fe72f865d405b6ce"
server_key_iv: "2c6da2bb7a952cb3a489385162db2f5d"
server_key_version: "encryptionkey@0"
I20260806 20:30:58.848989 9930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/wal/instance:
uuid: "22fb08d988f443438a0313dbef847242"
format_stamp: "Formatted at 2026-08-06 20:30:58 on dist-test-slave-6k22"
server_key: "63edf23310269bb8fe72f865d405b6ce"
server_key_iv: "2c6da2bb7a952cb3a489385162db2f5d"
server_key_version: "encryptionkey@0"
I20260806 20:30:58.854959 9930 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 20:30:58.858937 9945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:58.860334 9930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 20:30:58.860520 9930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/wal
uuid: "22fb08d988f443438a0313dbef847242"
format_stamp: "Formatted at 2026-08-06 20:30:58 on dist-test-slave-6k22"
server_key: "63edf23310269bb8fe72f865d405b6ce"
server_key_iv: "2c6da2bb7a952cb3a489385162db2f5d"
server_key_version: "encryptionkey@0"
I20260806 20:30:58.860747 9930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:58.886194 9930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:58.890776 9930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:58.891064 9930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:58.909786 9930 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.253:39773
I20260806 20:30:58.909808 9998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.253:39773 every 8 connection(s)
I20260806 20:30:58.911273 9930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/data/info.pb
I20260806 20:30:58.913172 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 9930
I20260806 20:30:58.913383 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-1/wal/instance
I20260806 20:30:58.913992 17847 external_mini_cluster.cc:1428] Setting key 49c7d8193a0cb192d458d24ffe2f9ce4
I20260806 20:30:58.915931 9999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:58.926103 9999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:30:58 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048258, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.109.252@KRBTEST.COM; defaulting to no policy
I20260806 20:30:58.950604 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.929627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.17.109.254:46385)
0806 20:30:58.930281 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:58.930290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:58.931320 (+ 1030us) server_negotiation.cc:407] Connection header received
0806 20:30:58.933366 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:58.933375 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:58.933476 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:58.933632 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:58.935169 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.936510 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.938188 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.938957 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.939133 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:58.942971 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:58.942981 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:58.942989 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:58.943040 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:58.946881 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:58.947708 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:58.947717 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:58.947723 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:58.947898 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:58.948406 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:58.948413 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:58.948419 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:58.948829 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:58.949008 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:58.949903 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 20:30:58.950177 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":284,"threads_started":1}
I20260806 20:30:58.951409 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.930282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46385 (local address 127.0.0.1:41804)
0806 20:30:58.931075 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.931122 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:58.933070 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:58.933852 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:58.933860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:58.934401 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:58.934947 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:58.934964 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.936754 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.936759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:58.938025 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:58.938039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.939208 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.939214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:58.939429 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:58.940220 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:58.940243 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:58.942729 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:58.947049 (+ 4320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:58.947056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:58.947069 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:58.947527 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:58.948047 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:58.948051 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:58.948054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:58.948269 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:58.948998 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:58.949008 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:58.949494 (+ 486us) client_negotiation.cc:769] Sending connection context
0806 20:30:58.949743 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 20:30:58.950146 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":346,"threads_started":1}
I20260806 20:30:58.953480 9999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.109.252@KRBTEST.COM" created.
Aug 06 20:30:58 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048258, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
I20260806 20:30:58.971611 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.954578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41302)
0806 20:30:58.954772 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.954798 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:58.954952 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:58.955909 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:58.955912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:58.956322 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:58.956471 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:58.956479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.958702 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.958706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:58.959627 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:58.959640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.960840 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.960844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:58.961004 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:58.961656 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:58.961673 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:58.964100 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:58.968789 (+ 4689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:58.968795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:58.968797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:58.969202 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:58.969654 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:58.969660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:58.969663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:58.969872 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:58.970666 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:58.970671 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:58.970847 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 20:30:58.971009 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 20:30:58.971276 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":1}
I20260806 20:30:58.972270 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.954571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41302 (local address 127.17.109.253:39773)
0806 20:30:58.955298 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:58.955309 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:58.955346 (+ 37us) server_negotiation.cc:407] Connection header received
0806 20:30:58.955410 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:58.955422 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:58.955745 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:58.955920 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:58.956653 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.958556 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.959760 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:58.960688 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:58.960826 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:58.964342 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:58.964359 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:58.964379 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:58.964422 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:58.968630 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:58.969338 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:58.969350 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:58.969356 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:58.969514 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:58.970038 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:58.970043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:58.970045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:58.970510 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:58.970712 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:58.971232 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 20:30:58.971585 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":490,"thread_start_us":331,"threads_started":1}
I20260806 20:30:58.975117 9999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:30:58.976593 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.975938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60878)
0806 20:30:58.976210 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.976367 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260806 20:30:58.976920 9947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
I20260806 20:30:58.978703 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:58.978159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60884)
0806 20:30:58.978386 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:58.978509 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260806 20:30:58.978995 9999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.109.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab.
Entry for principal kudu/127.17.109.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab.
I20260806 20:30:58.999491 17847 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
/tmp/dist-test-taskcvsjCP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.252:35091
--webserver_interface=127.17.109.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab
--principal=kudu/127.17.109.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.109.212:41243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.109.254:46385,127.17.109.253:39773,127.17.109.252:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 20:30:59.028332 9999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 1
I20260806 20:30:59.029610 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.029127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60890)
0806 20:30:59.029331 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.029448 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260806 20:30:59.030768 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.030366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60900)
0806 20:30:59.030522 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.030608 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 20:30:59.030988 9999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
I20260806 20:30:59.085300 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } attempt: 5
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048258, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
I20260806 20:30:59.098790 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.086297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41316 (local address 127.17.109.253:39773)
0806 20:30:59.086490 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.086496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.086517 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:30:59.086699 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.086703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.086789 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.086897 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.087723 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.088417 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.089595 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.090061 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.090203 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.093459 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.093464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.093467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.093500 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.096250 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.096904 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.096912 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.096915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.097043 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.097406 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.097412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.097415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.097690 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.097841 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.098222 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.098468 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 20:30:59.098809 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.086133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41316)
0806 20:30:59.086361 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.086386 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:59.086521 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:59.086932 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:59.086936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:59.087353 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:59.087522 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.087532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.088551 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.088558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.089432 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.089444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.090212 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.090217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.090477 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.091146 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:59.091165 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:59.093264 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:59.096390 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.096395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.096400 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.096784 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.097155 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.097160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.097164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.097302 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.097812 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:59.097816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:59.097972 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 20:30:59.098125 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 20:30:59.098451 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 20:30:59.099198 9999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 2
I20260806 20:30:59.100160 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.099761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60912)
0806 20:30:59.099920 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.100012 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 20:30:59.100203 9918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:30:59.101107 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.100746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60918)
0806 20:30:59.100884 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.100963 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 20:30:59.101265 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.100856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60922)
0806 20:30:59.101010 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.101096 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 20:30:59.101366 9999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
I20260806 20:30:59.102351 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.102014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60926)
0806 20:30:59.102141 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.102219 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 20:30:59.102576 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
I20260806 20:30:59.153800 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 1
I20260806 20:30:59.155073 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.154495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60938)
0806 20:30:59.154739 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.154860 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260806 20:30:59.156222 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.155842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60940)
0806 20:30:59.156000 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.156090 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 20:30:59.156476 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
I20260806 20:30:59.192627 9999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 3
I20260806 20:30:59.193917 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.193393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60948)
0806 20:30:59.193603 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.193745 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 20:30:59.194950 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.194526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60952)
0806 20:30:59.194758 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.194836 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260806 20:30:59.195134 9999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
I20260806 20:30:59.224731 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 2
I20260806 20:30:59.226078 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.225485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60964)
0806 20:30:59.225756 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.225871 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260806 20:30:59.227402 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.226896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60976)
0806 20:30:59.227093 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.227204 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 20:30:59.227648 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
I20260806 20:30:59.333889 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 3
I20260806 20:30:59.335246 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.334677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60984)
0806 20:30:59.334972 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.335067 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 20:30:59.336441 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.336022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60988)
0806 20:30:59.336249 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.336342 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260806 20:30:59.336632 9918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
I20260806 20:30:59.338335 9999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 4
I20260806 20:30:59.339336 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.338837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60994)
0806 20:30:59.339066 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.339168 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 20:30:59.340333 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.339974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:60996)
0806 20:30:59.340120 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.340203 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260806 20:30:59.340587 9999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.109.252:35091: Network error: Client connection negotiation failed: client connection to 127.17.109.252:35091: connect: Connection refused (error 111)
W20260806 20:30:59.343446 10007 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 20:30:59.343830 10007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 20:30:59.343955 10007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 20:30:59.344081 10007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 20:30:59.351332 10007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 20:30:59.351436 10007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 20:30:59.351575 10007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 20:30:59.351672 10007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 20:30:59.351759 10007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 20:30:59.361920 10007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.109.212:41243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.109.254:46385,127.17.109.253:39773,127.17.109.252:35091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/krb5kdc/kudu_127.17.109.252.keytab
--principal=kudu/127.17.109.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.109.252:35091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.109.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 06 Aug 2026 19:07:42 UTC on ccycloud.quasar-cuhmbo.root.comops.site
build id 4571034
ASAN enabled
I20260806 20:30:59.363252 10007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 20:30:59.364845 10007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 20:30:59.377590 10014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:59.377626 10013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 20:30:59.378067 10016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 20:30:59.378512 10007 server_base.cc:1034] running on GCE node
I20260806 20:30:59.379256 10007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 20:30:59.380697 10007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 20:30:59.381940 10007 hybrid_clock.cc:648] HybridClock initialized: now 1786048259381865 us; error 53 us; skew 500 ppm
I20260806 20:30:59.382305 10007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048259, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 20:30:59.385506 10007 init.cc:377] Logged in from keytab as kudu/127.17.109.252@KRBTEST.COM (short username kudu)
I20260806 20:30:59.386858 10007 webserver.cc:460] Webserver started at http://127.17.109.252:44811/ using document root <none> and password file <none>
I20260806 20:30:59.387494 10007 fs_manager.cc:359] Metadata directory not provided
I20260806 20:30:59.387633 10007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 20:30:59.387975 10007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 20:30:59.390882 10007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/data/instance:
uuid: "57112df22a7a4604baf38bcac32d64f5"
format_stamp: "Formatted at 2026-08-06 20:30:59 on dist-test-slave-6k22"
server_key: "70e9c45d2bbd8e2df3b0688535e9a053"
server_key_iv: "745f29dd84cea5595f6c4f8eee3b8fc4"
server_key_version: "encryptionkey@0"
I20260806 20:30:59.391752 10007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/wal/instance:
uuid: "57112df22a7a4604baf38bcac32d64f5"
format_stamp: "Formatted at 2026-08-06 20:30:59 on dist-test-slave-6k22"
server_key: "70e9c45d2bbd8e2df3b0688535e9a053"
server_key_iv: "745f29dd84cea5595f6c4f8eee3b8fc4"
server_key_version: "encryptionkey@0"
I20260806 20:30:59.397068 10007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 20:30:59.400697 10024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 20:30:59.402079 10007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 20:30:59.402281 10007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/data,/tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/wal
uuid: "57112df22a7a4604baf38bcac32d64f5"
format_stamp: "Formatted at 2026-08-06 20:30:59 on dist-test-slave-6k22"
server_key: "70e9c45d2bbd8e2df3b0688535e9a053"
server_key_iv: "745f29dd84cea5595f6c4f8eee3b8fc4"
server_key_version: "encryptionkey@0"
I20260806 20:30:59.402550 10007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 20:30:59.439529 10007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 20:30:59.443882 10007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 20:30:59.444116 10007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 20:30:59.462604 10007 rpc_server.cc:307] RPC server started. Bound to: 127.17.109.252:35091
I20260806 20:30:59.462612 10079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.109.252:35091 every 8 connection(s)
I20260806 20:30:59.464116 10007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/data/info.pb
I20260806 20:30:59.468150 10080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 20:30:59.474080 17847 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu as pid 10007
I20260806 20:30:59.474303 17847 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskcvsjCP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786047878953513-17847-0/minicluster-data/master-2/wal/instance
I20260806 20:30:59.474895 17847 external_mini_cluster.cc:1428] Setting key 5ac3ee770197a407d99a42af1fc38a79
I20260806 20:30:59.474973 10080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048259, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
I20260806 20:30:59.495270 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.478105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.17.109.254:46385)
0806 20:30:59.479948 (+ 1843us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.479956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.479977 (+ 21us) server_negotiation.cc:407] Connection header received
0806 20:30:59.480408 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.480413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.480490 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.480603 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.482212 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.482934 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.484482 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.484887 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.485020 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.488681 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.488688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.488693 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.488747 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.491988 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.492765 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.492770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.492772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.492897 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.493335 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.493340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.493342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.493603 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.493787 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.494654 (+ 867us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.494886 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1723,"thread_start_us":1598,"threads_started":1}
I20260806 20:30:59.496297 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.476894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.17.109.252:35091)
0806 20:30:59.477712 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.477727 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.477774 (+ 47us) server_negotiation.cc:407] Connection header received
0806 20:30:59.480129 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.480149 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.480431 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.480644 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.482048 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.483250 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.484580 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.485106 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.485324 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.489656 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.489665 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.489677 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.489704 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.492432 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.493086 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.493098 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.493100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.493231 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.493599 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.493606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.493609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.494184 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.494558 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.495169 (+ 611us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.495719 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":584,"spinlock_wait_cycles":19712,"thread_start_us":368,"threads_started":1}
I20260806 20:30:59.496347 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.478041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.254:46385 (local address 127.0.0.1:41820)
0806 20:30:59.479013 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.479086 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:59.480140 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:59.481087 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:59.481092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:59.481669 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:59.482028 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.482041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.483095 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.483105 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.484318 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.484334 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.485118 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.485125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.485316 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.486073 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:59.486104 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:59.488483 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:59.492139 (+ 3656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.492150 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.492162 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.492626 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.493007 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.493011 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.493015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.493204 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.493730 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:59.493738 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:59.494186 (+ 448us) client_negotiation.cc:769] Sending connection context
0806 20:30:59.494537 (+ 351us) client_negotiation.cc:240] Negotiation successful
0806 20:30:59.494994 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":42,"thread_start_us":329,"threads_started":1}
I20260806 20:30:59.498801 10080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048259, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.254@KRBTEST.COM
I20260806 20:30:59.513039 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.499650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.253:39773 (local address 127.0.0.1:41326)
0806 20:30:59.499897 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.499916 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:59.500008 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:59.500440 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:59.500443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:59.500763 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:59.500897 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.500905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.502039 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.502045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.503027 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.503038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.504030 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.504036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.504282 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.504954 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:59.504970 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:59.506955 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:59.510391 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.510396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.510398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.510778 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.511223 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.511226 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.511229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.511391 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.512045 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:59.512050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:59.512219 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 20:30:59.512363 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 20:30:59.512658 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260806 20:30:59.513084 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.499923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41326 (local address 127.17.109.253:39773)
0806 20:30:59.500105 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.500109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.500124 (+ 15us) server_negotiation.cc:407] Connection header received
0806 20:30:59.500206 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.500211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.500285 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.500389 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.501045 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.501903 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.503203 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.503900 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.503997 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.507203 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.507209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.507212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.507246 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.510238 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.510902 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.510908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.510914 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.511087 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.511525 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.511532 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.511535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.511917 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.512085 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.512456 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.512704 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 20:30:59.513837 9918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 4
I20260806 20:30:59.514431 10080 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } has no permanent_uuid. Determining permanent_uuid...
I20260806 20:30:59.516691 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.499543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.17.109.254:46385)
0806 20:30:59.499855 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.499861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.499881 (+ 20us) server_negotiation.cc:407] Connection header received
0806 20:30:59.500004 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.500009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.500098 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.500181 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.501469 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.502246 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.503783 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.504358 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.504505 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.509684 (+ 5179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.509694 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.509700 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.509744 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.512619 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.513391 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.513395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.513398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.513558 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.514147 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.514155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.514161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.514559 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.514875 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.516173 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.516458 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":137}
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048258, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.254@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.109.253@KRBTEST.COM
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048259, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.252@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
I20260806 20:30:59.532253 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.514439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:32776)
0806 20:30:59.514797 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.514819 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:59.514937 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:59.515340 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:59.515346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:59.515963 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:59.516321 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.516334 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.519019 (+ 2685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.519026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.520752 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.520768 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.522044 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.522051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.522257 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.522940 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:59.522965 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:59.525704 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:59.529551 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.529556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.529561 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.530117 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.530551 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.530557 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.530562 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.530735 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.531418 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:59.531422 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:59.531615 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 20:30:59.531748 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 20:30:59.531955 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":44}
I20260806 20:30:59.532486 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.514564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.17.109.252:35091)
0806 20:30:59.514712 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.514719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.514882 (+ 163us) server_negotiation.cc:407] Connection header received
0806 20:30:59.515062 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.515067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.515170 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.516272 (+ 1102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.516461 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.518875 (+ 2414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.520930 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.521878 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.522039 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.525942 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.525951 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.525956 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.525997 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.529423 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.530259 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.530268 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.530273 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.530430 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.530888 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.530893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.530897 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.531272 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.531433 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.531897 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.532129 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 20:30:59.537595 9918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627: Bootstrap starting.
I20260806 20:30:59.538797 10085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.515328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:32790)
0806 20:30:59.519169 (+ 3841us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.519190 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:59.519288 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:59.524413 (+ 5125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:59.524417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:59.524887 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:59.525078 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.525090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.526664 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.526672 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.528252 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.528269 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.529504 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.529509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.529717 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.530264 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:59.530284 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:59.532891 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:59.536399 (+ 3508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.536405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.536409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.536767 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.537106 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.537112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.537115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.537242 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.537761 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:59.537766 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:59.537904 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 20:30:59.538030 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 20:30:59.538345 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3694,"spinlock_wait_cycles":45440}
I20260806 20:30:59.538905 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.515236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.17.109.252:35091)
0806 20:30:59.524097 (+ 8861us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.524105 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.524131 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:30:59.524183 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.524189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.524279 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.524388 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.525282 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.526516 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.528419 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.529165 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.529291 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.533103 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.533107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.533110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.533135 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.536282 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.536882 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.536887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.536891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.537006 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.537357 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.537362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.537366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.537643 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.537789 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.538141 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.538484 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":8720,"thread_start_us":8523,"threads_started":1}
I20260806 20:30:59.541997 9918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:59.542104 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.518521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41338 (local address 127.17.109.253:39773)
0806 20:30:59.518675 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.518679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.518901 (+ 222us) server_negotiation.cc:407] Connection header received
0806 20:30:59.519115 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.519122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.519197 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.519281 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.520972 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.521635 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.523286 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.524047 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.524190 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.529459 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.529468 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.529475 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.529520 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.538644 (+ 9124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.539721 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.539726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.539730 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.539897 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.540370 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.540375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.540378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.540747 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.541035 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.541496 (+ 461us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.541749 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 20:30:59.543114 9918 log.cc:824] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:59.544433 10080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5: Bootstrap starting.
I20260806 20:30:59.545806 9918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627: No bootstrap required, opened a new log
I20260806 20:30:59.547655 10080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:59.548940 10080 log.cc:824] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:59.551091 17847 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 20:30:59.552112 10080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5: No bootstrap required, opened a new log
I20260806 20:30:59.554280 9889 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5a5aef0fc58444878d4398a486c71627"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41822
I20260806 20:30:59.554600 9918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } }
I20260806 20:30:59.554921 9918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a5aef0fc58444878d4398a486c71627, State: Initialized, Role: FOLLOWER
I20260806 20:30:59.555338 9918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } }
I20260806 20:30:59.556322 10090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.556720 10090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:59.557427 9918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:59.557526 9889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 20:30:59.557740 9889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:59.561452 9889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } }
I20260806 20:30:59.562872 10080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } }
I20260806 20:30:59.563292 10080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57112df22a7a4604baf38bcac32d64f5, State: Initialized, Role: FOLLOWER
I20260806 20:30:59.563863 10080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } }
I20260806 20:30:59.564651 9889 leader_election.cc:288] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [CANDIDATE]: Term 1 election: Requested vote from peers 22fb08d988f443438a0313dbef847242 (127.17.109.253:39773), 57112df22a7a4604baf38bcac32d64f5 (127.17.109.252:35091)
I20260806 20:30:59.581030 10097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.582353 10080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:59.582358 10054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a5aef0fc58444878d4398a486c71627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "57112df22a7a4604baf38bcac32d64f5"
I20260806 20:30:59.582906 10097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:59.582988 10054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:59.588455 10054 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a5aef0fc58444878d4398a486c71627 in term 1.
I20260806 20:30:59.589542 9862 leader_election.cc:302] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57112df22a7a4604baf38bcac32d64f5, 5a5aef0fc58444878d4398a486c71627; no voters:
I20260806 20:30:59.590624 10090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [term 1 FOLLOWER]: Leader election won for term 1
W20260806 20:30:59.592870 10108 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 20:30:59.593103 10108 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 20:30:59.603724 10090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [term 1 LEADER]: Becoming Leader. State: Replica: 5a5aef0fc58444878d4398a486c71627, State: Running, Role: LEADER
I20260806 20:30:59.604830 10090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } }
W20260806 20:30:59.608592 9866 tablet.cc:2369] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:59.618810 10110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a5aef0fc58444878d4398a486c71627. Latest consensus state: current_term: 1 leader_uuid: "5a5aef0fc58444878d4398a486c71627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.619282 10110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:59.620658 9842 master_runner.cc:186] Error getting master registration for 127.17.109.253:39773: OK, instance_id { permanent_uuid: "22fb08d988f443438a0313dbef847242" instance_seqno: 1786048258904414 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 20:30:59.639819 9999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } attempt: 5
I20260806 20:30:59.640568 9842 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 20:30:59.640300 10116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 20:30:59.643172 10116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 20:30:59.650447 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.600939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.17.109.254:46385)
0806 20:30:59.601197 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.601207 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.616015 (+ 14808us) server_negotiation.cc:407] Connection header received
0806 20:30:59.616640 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.616647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.616742 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.616828 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.621975 (+ 5147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.623337 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.627328 (+ 3991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.628143 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.628253 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.634203 (+ 5950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.634214 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.634223 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.634322 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.640482 (+ 6160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.643071 (+ 2589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.643077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.643111 (+ 34us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.643484 (+ 373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.645784 (+ 2300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.645792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.645795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.646644 (+ 849us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.647187 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.649039 (+ 1852us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.649586 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 20:30:59.651769 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.600912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.17.109.252:35091)
0806 20:30:59.601449 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.601459 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.609519 (+ 8060us) server_negotiation.cc:407] Connection header received
0806 20:30:59.611178 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.611187 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.611321 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.611414 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.620843 (+ 9429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.622065 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.625405 (+ 3340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.626051 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.626165 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.635307 (+ 9142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.635316 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.635322 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.635369 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.639393 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.642379 (+ 2986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.642386 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.642390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.642560 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.644854 (+ 2294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.644860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.644863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.645301 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.645587 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.651040 (+ 5453us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.651347 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343}
Aug 06 20:30:59 dist-test-slave-6k22 krb5kdc[9828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786048258, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.109.253@KRBTEST.COM for kudu/127.17.109.252@KRBTEST.COM
I20260806 20:30:59.655722 10004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.610821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41350 (local address 127.17.109.253:39773)
0806 20:30:59.610984 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.610989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.620194 (+ 9205us) server_negotiation.cc:407] Connection header received
0806 20:30:59.620388 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.620393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.620482 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.620581 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.622958 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.624054 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.627954 (+ 3900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.628753 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.628867 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.645322 (+ 16455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.645330 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.645335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.645381 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.649995 (+ 4614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.651988 (+ 1993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.651996 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.652000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.652188 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.653130 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.653136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.653140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.653579 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.653836 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.654626 (+ 790us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.655037 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 20:30:59.675482 10054 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [term 1 FOLLOWER]: Refusing update from remote peer 5a5aef0fc58444878d4398a486c71627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:59.677105 10090 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 20:30:59.696746 10030 tablet.cc:2369] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 20:30:59.703641 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.640496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.109.252:35091 (local address 127.0.0.1:32798)
0806 20:30:59.640926 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0806 20:30:59.640952 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 20:30:59.641101 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 20:30:59.642267 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 20:30:59.642274 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 20:30:59.642874 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0806 20:30:59.643115 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.643130 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.645344 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.645350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.647120 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 20:30:59.647141 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.648660 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.648666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 20:30:59.649004 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.649708 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 20:30:59.649739 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 20:30:59.659614 (+ 9875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 20:30:59.692597 (+ 32983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.692607 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.692615 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.693557 (+ 942us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.700430 (+ 6873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 20:30:59.700441 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 20:30:59.700448 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 20:30:59.700784 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 20:30:59.701948 (+ 1164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 20:30:59.701957 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 20:30:59.702227 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 20:30:59.702455 (+ 228us) client_negotiation.cc:240] Negotiation successful
0806 20:30:59.702943 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282}
I20260806 20:30:59.705433 10084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:30:59.641365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.17.109.252:35091)
0806 20:30:59.641569 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.641578 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.641604 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:30:59.641672 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.641681 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.641787 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.641906 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.643389 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.645026 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.647345 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:30:59.648265 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:30:59.648409 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:30:59.659950 (+ 11541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 20:30:59.659961 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 20:30:59.659970 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 20:30:59.660026 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 20:30:59.692191 (+ 32165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.693811 (+ 1620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.693821 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.693828 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.694053 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 20:30:59.700988 (+ 6935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 20:30:59.700998 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 20:30:59.701005 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 20:30:59.701632 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 20:30:59.701852 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0806 20:30:59.704471 (+ 2619us) server_negotiation.cc:299] Negotiation successful
0806 20:30:59.704850 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 20:30:59.708648 10097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a5aef0fc58444878d4398a486c71627. Latest consensus state: current_term: 1 leader_uuid: "5a5aef0fc58444878d4398a486c71627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.708998 10097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:59.712817 9999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242: Bootstrap starting.
I20260806 20:30:59.715788 10110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a5aef0fc58444878d4398a486c71627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.716099 10110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:59.718845 10116 catalog_manager.cc:1347] Generated new cluster ID: a9b7b2a4b065423b8c8a6144212a840f
I20260806 20:30:59.718964 10116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 20:30:59.722560 9999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242: Neither blocks nor log segments found. Creating new log.
I20260806 20:30:59.727258 9999 log.cc:824] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242: Log is configured to *not* fsync() on all Append() calls
I20260806 20:30:59.734213 10097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a5aef0fc58444878d4398a486c71627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.734572 10097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:59.742619 9999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242: No bootstrap required, opened a new log
I20260806 20:30:59.778297 9999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } }
I20260806 20:30:59.778991 9999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fb08d988f443438a0313dbef847242, State: Initialized, Role: FOLLOWER
I20260806 20:30:59.779772 9999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } }
I20260806 20:30:59.784785 10135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.787438 10116 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 20:30:59.789324 10116 catalog_manager.cc:1504] Loading token signing keys...
I20260806 20:30:59.797628 10135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:59.787352 9973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5a5aef0fc58444878d4398a486c71627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "22fb08d988f443438a0313dbef847242"
I20260806 20:30:59.788079 9972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [term 0 FOLLOWER]: Advancing to term 1
I20260806 20:30:59.787262 9999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 20:30:59.802358 9972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [term 1 FOLLOWER]: Refusing update from remote peer 5a5aef0fc58444878d4398a486c71627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 20:30:59.804014 10090 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 20:30:59.809302 9973 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a5aef0fc58444878d4398a486c71627 in term 1.
I20260806 20:30:59.814982 10116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627: Generated new TSK 0
I20260806 20:30:59.816324 10116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 20:30:59.843377 10136 catalog_manager.cc:1259] Loaded cluster ID: a9b7b2a4b065423b8c8a6144212a840f
I20260806 20:30:59.843497 10136 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5: loading cluster ID for follower catalog manager: success
I20260806 20:30:59.793280 10007 master_runner.cc:186] Error getting master registration for 127.17.109.253:39773: OK, instance_id { permanent_uuid: "22fb08d988f443438a0313dbef847242" instance_seqno: 1786048258904414 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 20:30:59.850944 10007 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 20:30:59.860972 10136 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5: acquiring CA information for follower catalog manager: success
I20260806 20:30:59.867197 10136 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 57112df22a7a4604baf38bcac32d64f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 20:30:59.892558 10141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a5aef0fc58444878d4398a486c71627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.893117 10141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:59.893513 10135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a5aef0fc58444878d4398a486c71627. Latest consensus state: current_term: 1 leader_uuid: "5a5aef0fc58444878d4398a486c71627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.893785 10135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 20:30:59.895635 10110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a5aef0fc58444878d4398a486c71627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a5aef0fc58444878d4398a486c71627" member_type: VOTER last_known_addr { host: "127.17.109.254" port: 46385 } } peers { permanent_uuid: "22fb08d988f443438a0313dbef847242" member_type: VOTER last_known_addr { host: "127.17.109.253" port: 39773 } } peers { permanent_uuid: "57112df22a7a4604baf38bcac32d64f5" member_type: VOTER last_known_addr { host: "127.17.109.252" port: 35091 } } }
I20260806 20:30:59.895918 10110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a5aef0fc58444878d4398a486c71627 [sys.catalog]: This master's current role is: LEADER
I20260806 20:30:59.928027 10158 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260806 20:30:59.967511 17847 mini_kdc.cc:348] Time spent kdestroy: real 0.133s user 0.000s sys 0.035s
W20260806 20:30:59.986634 10086 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:59.978220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.17.109.254:46385)
0806 20:30:59.978648 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.978655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.984431 (+ 5776us) server_negotiation.cc:407] Connection header received
0806 20:30:59.984966 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.984972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.985062 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.985138 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.986349 (+ 1211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41848: BlockingRecv error: recv got EOF from 127.0.0.1:41848 (error 108)
Metrics: {"server-negotiator.queue_time_us":264}
W20260806 20:30:59.989117 10084 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:59.978411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32814 (local address 127.17.109.252:35091)
0806 20:30:59.978642 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.978651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.987445 (+ 8794us) server_negotiation.cc:407] Connection header received
0806 20:30:59.987715 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.987723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.987843 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.987917 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.988883 (+ 966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32814: BlockingRecv error: recv got EOF from 127.0.0.1:32814 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 20:30:59.991446 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:30:59.977205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.17.109.253:39773)
0806 20:30:59.977413 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 20:30:59.977420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:30:59.989891 (+ 12471us) server_negotiation.cc:407] Connection header received
0806 20:30:59.990133 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:30:59.990138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:30:59.990238 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:30:59.990314 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:30:59.991226 (+ 912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41362: BlockingRecv error: recv got EOF from 127.0.0.1:41362 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260806 20:31:00.002967 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.000335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41378 (local address 127.17.109.253:39773)
0806 20:31:00.000577 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.000585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.001217 (+ 632us) server_negotiation.cc:407] Connection header received
0806 20:31:00.001481 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.001488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.001617 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.001745 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.002681 (+ 936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41378: BlockingRecv error: recv got EOF from 127.0.0.1:41378 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260806 20:31:00.010924 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.008597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41394 (local address 127.17.109.253:39773)
0806 20:31:00.008811 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.008818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.009384 (+ 566us) server_negotiation.cc:407] Connection header received
0806 20:31:00.009604 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.009610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.009695 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.009794 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.010721 (+ 927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41394: BlockingRecv error: recv got EOF from 127.0.0.1:41394 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260806 20:31:00.019547 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.016556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41396 (local address 127.17.109.253:39773)
0806 20:31:00.016774 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.016781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.017566 (+ 785us) server_negotiation.cc:407] Connection header received
0806 20:31:00.017801 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.017806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.017910 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.018020 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.019253 (+ 1233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41396: BlockingRecv error: recv got EOF from 127.0.0.1:41396 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260806 20:31:00.029412 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.026881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41402 (local address 127.17.109.253:39773)
0806 20:31:00.027114 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.027120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.027686 (+ 566us) server_negotiation.cc:407] Connection header received
0806 20:31:00.027945 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.027953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.028065 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.028207 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.029126 (+ 919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41402: BlockingRecv error: recv got EOF from 127.0.0.1:41402 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260806 20:31:00.047236 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.044522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41416 (local address 127.17.109.253:39773)
0806 20:31:00.044942 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.044948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.045538 (+ 590us) server_negotiation.cc:407] Connection header received
0806 20:31:00.045747 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.045755 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.045868 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.046003 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.046941 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41416: BlockingRecv error: recv got EOF from 127.0.0.1:41416 (error 108)
Metrics: {"server-negotiator.queue_time_us":283}
W20260806 20:31:00.069796 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.067229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.17.109.253:39773)
0806 20:31:00.067439 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.067446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.068099 (+ 653us) server_negotiation.cc:407] Connection header received
0806 20:31:00.068318 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.068326 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.068435 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.068566 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.069508 (+ 942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41424: BlockingRecv error: recv got EOF from 127.0.0.1:41424 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260806 20:31:00.110584 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.107931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.17.109.253:39773)
0806 20:31:00.108147 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.108154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.108769 (+ 615us) server_negotiation.cc:407] Connection header received
0806 20:31:00.109011 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.109019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.109130 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.109267 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.110285 (+ 1018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41432: BlockingRecv error: recv got EOF from 127.0.0.1:41432 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 20:31:00.187686 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.185024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41434 (local address 127.17.109.253:39773)
0806 20:31:00.185287 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.185297 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.185959 (+ 662us) server_negotiation.cc:407] Connection header received
0806 20:31:00.186183 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.186189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.186290 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.186405 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.187427 (+ 1022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41434: BlockingRecv error: recv got EOF from 127.0.0.1:41434 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260806 20:31:00.321900 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.319751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41450 (local address 127.17.109.253:39773)
0806 20:31:00.319966 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.319973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.320338 (+ 365us) server_negotiation.cc:407] Connection header received
0806 20:31:00.320518 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.320524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.320651 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.320799 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.321605 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41450: BlockingRecv error: recv got EOF from 127.0.0.1:41450 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260806 20:31:00.583743 10004 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 20:31:00.581525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.17.109.253:39773)
0806 20:31:00.581848 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:00.581853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:00.581979 (+ 126us) server_negotiation.cc:407] Connection header received
0806 20:31:00.582361 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:00.582367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:00.582441 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:00.582533 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 20:31:00.583462 (+ 929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41454: BlockingRecv error: recv got EOF from 127.0.0.1:41454 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
I20260806 20:31:00.932869 10158 catalog_manager.cc:1259] Loaded cluster ID: a9b7b2a4b065423b8c8a6144212a840f
I20260806 20:31:00.932998 10158 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242: loading cluster ID for follower catalog manager: success
I20260806 20:31:00.935891 10158 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242: acquiring CA information for follower catalog manager: success
I20260806 20:31:00.937738 10158 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 22fb08d988f443438a0313dbef847242: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 20:31:01.117146 10227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:31:01.108963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.17.109.253:39773)
0806 20:31:01.109907 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:01.109914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:01.109940 (+ 26us) server_negotiation.cc:407] Connection header received
0806 20:31:01.110027 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:01.110034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:01.110255 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:01.110422 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:31:01.110944 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.112354 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.113995 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.114809 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.114952 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:31:01.115534 (+ 582us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:31:01.116425 (+ 891us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:31:01.116603 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 20:31:01.116688 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 20:31:01.116781 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":788,"thread_start_us":632,"threads_started":1}
I20260806 20:31:01.131773 10233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:31:01.124149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.17.109.252:35091)
0806 20:31:01.125455 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:01.125460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:01.125475 (+ 15us) server_negotiation.cc:407] Connection header received
0806 20:31:01.125559 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:01.125563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:01.125705 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:01.125838 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:31:01.126384 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.127151 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.129330 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.129745 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.129839 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:31:01.130461 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:31:01.130921 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:31:01.131026 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 20:31:01.131266 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 20:31:01.131356 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":1107,"threads_started":1}
I20260806 20:31:01.151058 10233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:31:01.139307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.17.109.252:35091)
0806 20:31:01.143329 (+ 4022us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:01.143337 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:01.143903 (+ 566us) server_negotiation.cc:407] Connection header received
0806 20:31:01.144208 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:01.144213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:01.144328 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:01.144407 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:31:01.145338 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.146512 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.148917 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.149396 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.149555 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:31:01.150152 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:31:01.150296 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:31:01.150366 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 20:31:01.150620 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 20:31:01.150712 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3902}
I20260806 20:31:01.153342 10239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:31:01.138635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.17.109.254:46385)
0806 20:31:01.143319 (+ 4684us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:01.143327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:01.143351 (+ 24us) server_negotiation.cc:407] Connection header received
0806 20:31:01.143489 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:01.143498 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:01.143801 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:01.144664 (+ 863us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:31:01.144895 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.146196 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.149999 (+ 3803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.150705 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.150831 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:31:01.151632 (+ 801us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:31:01.152144 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:31:01.152252 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 20:31:01.152889 (+ 637us) server_negotiation.cc:299] Negotiation successful
0806 20:31:01.152989 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4555,"thread_start_us":4450,"threads_started":1}
I20260806 20:31:01.158736 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9842
I20260806 20:31:01.159521 10227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 20:31:01.144020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41462 (local address 127.17.109.253:39773)
0806 20:31:01.144203 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 20:31:01.144209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 20:31:01.151962 (+ 7753us) server_negotiation.cc:407] Connection header received
0806 20:31:01.152282 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 20:31:01.152290 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 20:31:01.152399 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 20:31:01.152475 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 20:31:01.153121 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.154371 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.157153 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 20:31:01.157751 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 20:31:01.157940 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 20:31:01.158537 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 20:31:01.158691 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 20:31:01.158807 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 20:31:01.159045 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 20:31:01.159152 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 20:31:01.159880 9913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:31:01.282909 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 9842
I20260806 20:31:01.316494 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 9930
I20260806 20:31:01.317729 9994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:31:01.435256 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 9930
I20260806 20:31:01.457975 17847 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu pid 10007
I20260806 20:31:01.459343 10074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260806 20:31:01.567574 17847 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskcvsjCP/build/asan/bin/kudu with pid 10007
2026-08-06T20:31:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5221 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9878 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (382549 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST